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ABSTRACT 

The GALAXY Language Arts Demonstration Program is a 
package of integrated curricular and instructional approaches that 
features the organization of instruction around themes presented 
through television broadcasts, children*s literature, classroom 
activities, and the use of interactive technology. During the GALAXY 

Project d. onstratiop phase for language arts in grades 3 through 5, 

classrooms in 37 schools were connected by an interactive satellite 
communications network. Seven successive themes were used, including 
such issues as fairness, personal privacy, and first impressions. 
Television broadcasts dramatized the GALAXY themes through a 
continuing show about the lives of a multicultural group of students. 
Specif ically, GALAXY classrooms utilized a core of six literature 
books for each of the seven themes, a take-home magazine (**The Second 
Story*') 1 a periodic response bulletin ("The Max"), a teacher's guide 
with suggestions on reading and writing strategies, and a wide choice 
of classroom activities for before, between, and after the two 
broadcasts for each theme. Measures of student learning were 
administered in all GALAXY classrooms and in two kinds of comparison 
classrooms, classrooms which were not using the program in GALAXY 
schools and others in demographical ly similar schools where GALAXY 
was not being implemented. Additionally repeated observations and 
interviews were carried out in five case study schools throughout the 
United States. Testing 2,826 students in GALAXY and comparison 
classrooms showed that GALAXY students outperformed others in reading 
and vocabulary growth, but GALAXY students did not outperform the 
others in writing achievement. Eleven appendixes provide technical 
information about the study, sample performance assessments, scoring 
guides, and results from a teacher survey. (Contains 8 figures, 37 
tables, and 19 additional tables in the appendixes.) (SLD) 
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Summary of Findings 



The GALAXY Language Arts Demonstration Program is a package of integrated 
curricular and instructional approaches. It features the organization of instruction 
around themes presented through television broadcasts, children's literature, 
classroom activities, and the use of interactive technology. 

During the GALAXY Classroom Project demonstration phase for language arts in 
grades three through five, classrooms in 37 schools were connected by an interactive 
satellite communications network. The demonstration took place during fourteen 
weeks in the spring of 1993. Each classroom was equipped with a fax machine, 
audioconferencing telephone (the "hoot 'n holler"), video cassette recorder (VCR), and 
television (TV), which were linked by Very Small Aperture Terminals (VSATs) to enable 
two-way voice and data commtmication and one-way television communication. 

The Far West Laboratory for Educational Research and Development carried out a 
multifaceted evaluation of the operation and impact of the program for this initial 
demonstration phase. The evaluation encompassed testing of student learning through 
both standardized measures and performance-based instruments, surveys of teacher 
and student attitudes and teacher practices, and teacher records of utilization of the 
GALAXY curriculiun. 

Measures of student learning in reading and writing were administered in all GALAXY 
classrooms and in two kinds of comparison classrooms, classrooms which were not 
using the program in GALAXY schools cind others in demographically similar schools 
where GALAXY was not being implemented. In addition, repeated observations and 
interviews were carried out in fiv,_ case study schools throughout the country, with 
shorter visits to several other schools. 

Thematic Curriculum 

The intellectual core of the GALAXY curriculum was the organization of all instruction 
around seven successive themes, each of them designed to be of genuine interest to 
GALAXY students. They included issues such as fairness, personal privacy, and first 
impressions. The success of this thematic structure was essential to the success of 
GALAXY, both for motivational reasons and for providing a coherent, integrated 
structure for the diverse GALAXY activities. 

Broadcasts. Literature, and Faxes 

The television broadcasts dramatized the GALAXY themes through a continuing show 
about the lives of a multicultural group of students, interacting together in an after- 
school community center. The story line was engaging and featured characters with 
whom students could identify. The open-ended themes in the programs were intended 
to stimulate critical thinking and discussion, unlike the traditional use of "instructional" 
television. Within the broadcasts, the characters modeled children communicating 
purposively through reading and writing. 
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GALAXY recommended children's literature to complement the themes and to engage 
the multicultural audience of learners; these books contrast significantly with traditional 
"basal" readers. The GALAXY books dealt with engaging subjects and characters with 
whom it is easy for GALAXY students to identify. Listening to the reading of good 
books provided a common basis for the sharing of perceptions and personal meaning. 

One of galaxy's unique advantages is the ability to engage students in interaction 
through its dedicated satellite network, classroom fax machines, and "hoot 'n holler" 
audioconferencing network. The fax, in particular, is integral to the motivational 
incentives of GALAXY'S approach to language learning. 

Reading Outcomes 

GALAXY students significantly outperformed comparable students in reading 
achievement, as measured by standardized test scores for two components of reading 
assessment. 'These results are based on testing 2,826 students in 93 GALAXY and 53 
comparison classrooms in January and May 1993, using the Vocabulary and Reading 
Comprehension subtests of the California Achievement Test, Fifth Edition (CAT/5). 

• These gains are similar for grades 3, 4, and 5, regardless of gender, ethnicity. Chapter 1 
eligibility, home language, and special education status. 

• Gains are similar regardless of how well students mitially performed on these measures. 
Vocabulary Results 

The results from the Vocabulary subtest showed a statistically significant greater gain 
for GALAXY than for comparison students. 

• GALAXY students had an average gain of 13.8 scale score points, while comparison 
students gained an average of 10.05 — a 37% greater gain for GALAXY students. GALAXY 
students performed above the expectation suggested by national norms by more than 307o. 

• When GALAXY and comparison classrooms are matched by rank from highest to lowest in 
terms of Vocabulary gain, GALAXY classrooms display consistently higher gains on 
Vocabulary scores. 

Reading Comprehension Results 

The Reading Comprehension subtest show-ed a statistically significant greater gain for 
GALAXY students than for comparison students. 

• GALAXY students had an average gain of 7.67 scale score points, whereas comparison 
students had an average gain of 3.13 points — GALAXY students made 145% greater gains 
than comparison students in Reading Comprehension. 

• When GALAXY and comparison classrooms are matched by rank from highest to lowest in 
terms of Reading Comprehension gain, GALAXY classrooms display higher gains at almost 
every point. 
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Factors associated with greater gain 

Other factors related to the intensity with which GALAXY was taught were associated 
with greater gains in test scores. 

• Students in the ten classrooms whose teachers used GALAXY as a replacement for their 
traditional language arts curriculum scored two-thirds better on the Vocabulary subtest 
(mean Vocabulary gain of 21.1) than students in the seventy-nine classrooms where 
GALAXY was a supplement (mean Vocabulary gain of 12.5). Both groups scored better than 
comparison students. 

• Average gains in classrooms where the fax machine was used two or three times per week 
(as recommended by GALAXY) were more than twice as high as for occasional use 
classrooms on Reading Comprehension (11.0 versus 4.0 scale score points) and almost 75% 
higher on Vocabulary (16.1 versus 9.3 scale score points). 

Writing Outcomes 

Writing is another important part of the Language Arts curriculum. GALAXY provided 
a unique context to stimulate student writing through faxing to real audiences and 
writing about themes that clearly engaged student interest. 

The evaluation measured one type of writing, persuasive writing, which required 
students to take a position on an issue and support it with evidence, a kind of writing 
that was closely allied with the GALAXY curriculum. Writing prompts, given to both 
GALAXY and comparison students, were used to elicit a pre/post measure of this 
writing ability, and they were supplemented with curriculum embedded writing 
performance tasks, given to GALAXY students only. 

• Both GALAXY and comparison students made statistically significant gains on the pre/ post 
writing prompts, but there was no significant difference in performance between the two 
groups. 

• GALAXY students showed statistically significant gains on the first three embedded 
performance assessments, but there was no significant gain when all fovir performance tasks 
were viewed together. However, performance assessment four was administered very close 
to the end of the school year, and student scores may well have reflected various 
distractions in the school environment. 

It is probable that GALAXY writing achievement will be enhanced as teachers learn to 
include more "process writing," the process of review and revision that is designed to 
develop writing skills, when they use the GALAXY curriculum. 

GALAXY reached its diverse multicultural learners in a way that generated 
extraordinary student enthusiasm. That enthusiasm was shared by teachers, who in 
numerous cases reported that the experience had brought new life to their professional 
work. GALAXY has had an impact beyond language arts; it has changed the way many 
teachers view the capabilities of their students. GALAXY'S unique interactive 
component, the use of the fax machine, fueled that enthusiasm, despite some technical 
problems. 
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I. Introduction 



Excitement, high expectations, lots of discussion, and plenty of writing comprised part 
of the GALAXY experience for the students who were in GALAXY Classrooms in the 
Spring of 1993. The GALAXY Classroom Project was developed as a nationwide reform 
effort to infuse new curricula into schools, to spark the interest of teachers and students, 
and to make a significant difference in the educational lives of students who 
traditionally have been labeled "at-risk." 

During the fourteen-week GALAXY Classroom Project demonstration phase for 
language arts in grades three through five, classrooms in 37 schools were cormected by 
an interactive satellite communications network. Each classroom was equipped with a 
fax machine, telephone, video cassette recorder (VCR), and television (TV), which were 
linked by Very Small Aperture Terminals (VSATs) to enable two-way voice and data 
communication and one-way television communication. 

GALAXY is more than the sum of its technology. GALAXY is a set of integrated 
curricular and instructional approaches. GALAXY Language Arts features the 
organization of instruction around themes, presented through television broadcasts, 
children's literature, classroom activities, and the use of technology. GALAXY 
classrooms utilized a core of six literature books for each of the seven themes, a take- 
home magazine {The SECOND STORY), a periodic response bulletin {The Max), a 
teacher's guide with suggestions on reading and writing strategies, and a wide choice of 
classroom activities for before, between, and after the two broadcasts for each theme. 

This document describes the findings from an evaluation of this demonstration phase of 
the language arts cycle for grades three through five (the first curricular area on which 
GALAXY focused). This evaluation report documents some of the changes, large and 
small, that took place in GALAXY Classrooms throughout the country. 
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n. Evaluation Design 



The GALAXY Classroom Project has a strategy for empowering learners: providing 
themes that have meaning for students, engaging students in real reading and writing 
tasks, and providing opportunities for the students to participate in problem solving, 
reflection, and reasoning in a language arts curriculum that employs innovative 
technologies. This curriculum, which reflects a shift in the goals and practices of 
education and a major change in teacher practice, requires a similar shift in evaluation 
methodology and practice. Researchers at Far West Laboratory for Educational 
Research and Development (FWL) developed a multi-method evaluation that gathered 
both quantitative and qualitative data to assess student outcomes, changes in teacher 
attitudes, and the implementation of the curriculum and its integral technology 
components. 



Quantitative Data 

Three quantitative measures of varying generalizability and purpose were used to 
assess students' progress in areas related to the GALAXY curriculum. These were 
norm-referenced tests, which provide a skills-oriented achievement measure; 
"persuasive" writing prompts, which showcased students' ability to choose a position 
and support it with evidence; and two pairs of curriculum-embedded performance 
assessments, which were designed to mirror closely the GALAXY curriculum. 

In addition, other measures gave evaluators a window on what participants were 
thinking and doing. Student perceptions of themselves as learners were measured both 
before and after their participation in the GALAXY classroom. Teachers' attitudes 
towards constructivist teaching were similarly measured. The "intensity of use" of the 
GALAXY curriculum was measured through bi-weekljr teacher activity logs. An end of 
year survey gave teachers the opportunity to rate the various components of GALAXY 
and to comment on how GALAXY had affected them and their students; about two- 
thirds of the GALAXY teachers returned the survey. 



Qualitative Data 

FWL researchers collected qualitative data by conducting a series of site visits to 
selected GALAXY sites. During the visits, researchers observed students and teachers 
engaged in GALAXY instruction and technology use, conducted structured interviews 
with teachers and principals, interviewed GALAXY students in small groups and 
informally, and analyzed student work in their GALAXY portfolios. 

Five schools were selected as case study sites and visited by FWL researchers for two to 
three days on three separate occasions during the fourteen-week cycle. In addition, a 
sixth school was visited twice and two others hosted a researcher on single occasions. 
These schools were selected for their geographic and ethnic diversity. Two of the 
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schools had significant bilingual populations, and all but one of the schools were 
eligible for Chapter 1 funds. In addition, schools were selected so that the sample 
would include both rural and urban sites. 



Comparison Groups 

The GALAXY evaluation employed two different comparison groups. The first group 
consisted of typically one third, fourth, or fifth grade classroom that was not 
participating in GALAXY in a school that had GALAXY classrooms. This comparison 
group allowed comparisons that held constant all school-level factors. The second 
comparison group encompassed classrooms that were not in GALAXY schools but were 
in the same district as GALAXY schools. This group enlarged the sample of comparison 
classrooms. The non-GALAXY schools were picked by the superintendent of the 
district in response to a request from FWL to select schools that were very similar to the 
GALAXY schools in the district. 

Sampling and other data selection 

Funding constraints dictated that not all measures could be scored for all students, so 
evaluators developed a random sampling plan that was designed to include students 
from every classroom. All students who took the CAT/5 pre-test were initially 
included in the evaluation; those that had only the pre-test or the post-test were 
dropped from subsequent analyses. From among all students who had taken the 
CAT/5 pre-test, approximately 87% of the writing prompts in each classroom were 
scored, and scores for those students with both pre-test and the post-test writing 
prompts were analyzed. For the performance assessments, which were designed to be 
paired, we included only classrooms that had administered the first and fourth and/or 
the second and third performance tests. The relevant sample sizes appear in the 
chapters on Reading and Writing. 

In addition, we omitted from the analysis those few sixth grade classrooms that 
participated, since the GALAXY curriculum was designed for grades 3 through 5; the 
two year-round schools since their schedules dictated that the FWL testing dates would 
be inappropriate; and schools that were not Chapter 1 participants. 
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111. Thematic Curriculum 



The GALAXY thematic curriculum is a big hit with both teachers and students. In this 
chapter, evidence from both teachers and students will be provided to support this 
finding. 

Conceptual Framework 

A conceptual framework underlies the GALAXY Classroom language arts curriculum. 
Its tenets are the following: (a) background knowledge plays a critical role in reading, 
writing, and learning; (b) curricula must address the linguistic, cognitive, sociocultural, 
and developmental dimensions of literacy; (c) children of diverse ethnic, linguistic, and 
economic backgroxinds must have equal access to experiences and knowledge at our 
nation's schools; (d) cultural diversity is a sT ~ngth (rather than a deficit) that children 
bring to school; and (e) the role of curriculum is to affirm, build upon, and extend the 
experiences of the students. 

Seven themes that have their foundation in this framework are part of the language arts 
curriculum. These themes are diverse, but all deal with social and interpersonal issues 
that affect all people. The seven themes are: (1) People Are More Than They Appear to 
Be, (2) Establishing Your Own Space, (3) Who Is a Hero?, (4) Be True to Yourself, (5) It's 
Not Fair, (6) People Need People, and (7) People Working Together Can Make a 
Difference. 



Teacher Reflections on the GALAXY Themes 

Responses to the end of year teacher survey reflect teachers' positive opinion of the 
GALAXY themes. Eighty-four percent of the teachers responded "very well" in answer 
to the question of how weU GALAXY helps students to relate the GALAXY themes to 
their own experience. Similarly, teachers thought that GALAXY did a good job in 
assisting students to develop an in-depth understanding of the themes (58% said 
GALAXY did this "very weU"). 



Table 1 Teacher Rating of Student Reaction to GALAXY Themes (Percent) 



; Since GALAXY befian, my student 








Alt; 


Relate the GALAXY themes to their own experience 


84.2 


14.9 


1 1.0 


0.0 


Develop an in-depth understanding of the seven themes 


58.4 


38.6 


1 3.0 


0.0 



Ninety- two percent of the teachers thought that the GALAXY themes had great 
educational value (only the broadcasts rated higher), and they were almost as sure that 
their students liked the themes (89% said their students liked the themes a lot). 
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Table 2. Teacher Rating of the GALAXY Themes 



Educational Value 





Number 


Percent 


Great Value 


96 


92.3 


r — 1 — 

Some Value 


8 


7.7 


Uttlo Value 


0 


0.0 


Did Not Use 


0 


0.0 



How Much Students Liked 





Number 


. Percent 


— — — =r 

Liked a Lot 


91 


89.2 


Liked Somewhat 


10 


9.8 


Did Not Uke ; . 

-Jr '■ ■ 


1 


1.0 




0 


0.0 



Teachers at the case study sites unanimously gave the seven GALAXY themes high 
marks. They were extremely impressed with the relevance of the themes to their 
students' lives, regardless of grade level. "Powerful," "motivating," and "relevant" 
were the terms they used repeatedly to describe these themes. Teachers frequently 
mentioned the themes when reflecting upon their GALAXY experience, as the following 
comments indicate; 

Teachers value the themes 

"Great Program! Our school has adopted the 7 themes schoolwide. We will 
do one them per six weeks, with the last 2 being combined." 

"I love the themes — very developmentally appropriate!" 

"The well thought out themes along with the corresponding literature have 
been fantastic!" 

"It has given me more of a structure. The themes were excellent & 
appropriate for 'today's' students. It enlightened me to great literature 
books." 

"It is generally very good! ... [GALAXY] gets teachers thinking in terms of 
themes, concepts & ideas, instead of isolated skills. GALAXY is really 
tops!!!" 

"I've really gotten into the themes— I really like them! Other themes have 
struck me as dumb or sappy. These were just right." 

Students love the themes 

■'All of my students loved GALAXY!! At the end of the year I asked my 
students if they could remember the GALAXY themes— they know every one 
of them." 

"The children love the thematic approach. It has improved their ability to 
focus on a problem." 
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Themes helped students to share and to think 

"My students related very well to the GALAXY classroom objectives because 
of the social structure of each theme. They were quite verbal and ready to 
share their experiences . " 

"Helped sensitize children to real issues. Lent sophisticated thought 
processes. " 

"Through ideas and discussions from the GALAXY curriculum, my students 
shared many feeling, problems and opinions that perhaps could not have been 
shared without suggestions that came out in each theme." 

"This program gets a Rave Review from me. I loved the themes you chose and 
the wonderful books selected to accompany them. But, watching my students 
loving to read and write more was extremely rewarding. I know my students 
showed a new height in being creative because the program greatly motivated 
them. They were kinder to their peers and I felt that I was a more effective 
teacher. " 

"The best part of GALAXY is the thematic approach. Literature selections are 
great — activities well done. Video component is the hook for students and 
pulls together theme." 

"I am more interested in developing themes for a whole language approach. 
While I have taught Literature based classes the last two years, GALAXY 
allowed a more whole language approach." 

"GALAXY has strengthened my teaching strategies. It has fit well into my 
'style' & given me a positive tool to foster a more positive interaction with my 
students 6* they with each other. It has allowed us to fit most everyday 
situations, other literature, true life (both past & present; fact & fantasy) into 
one of the themes, and has given a common, non-threatening way to deal with 
most situations. I love it and want to continue using GALA)^. Will it be 
up-dated & continued later on? Hope so. We're hooked! Thanks a million 
times. I'm sorry I didn't use the fax more, but the rest was used a lot & very 
much appreciated." 

Student comments on the GALAXY themes 

Students at the case study sites appeared not only to understand the themes 
but also to value highly the learning that came from exploring them. Their 
high regard for the themes is demonstrated in the answers they provided to 
FWL researchers when asked to imagine all the GALAXY activities in which 
they participated and then to think of the most important thing they learned 
while doing these activities. Their important GALAXY lessons included: 
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"Katie was in a wheel chair and nobody teased her." (People Are More Than 
They Appear to Be) 



"What I learned when Joey and Carl read Jannelle's diary and invaded her 
privacy was how you need to forgive your friends. Listen to yourself, and 
don't be hard on others because they did something bad to you. Give friends 
another chance." (Establishing Your Own Space) 

"I learned about ordinary people and how they are as important as 
superheroes. Ordinary people can be heroes too. " (Who Is a Hero?) 

“When you lie to yourself you hurt yourself, no one else. You get in more 
trouble when you lie. When you tell the truth, you don't get in as much 
trouble. It's an important value." (Be True to Yourself) 

"Don't keep things bottled up inside. Carl's father drank and he didn't tell 
nobody." (It's Not Fair) 

"If you don't have people in your life, then you can't learn or be alive. People 
write and read books and make cool GALAXY stuff, that's why we need 
people." (People Need People) 

"I used to think people can live without other people. This [GALAXY] proved 
to me that people need one another and that I need a lot of friends." (People 
need people) 

"People need other people. You can't build a house all by yourself." (People 
Need People) 

"All the time you have a problem, you should tell them and ask for advice. I 
don't like to tell about my problems, and tell people to 'bug off.' But then I 
don't have help. If you tell people, then you get lots of suggestions. All the 
characters at THE HOUSE had problems and we helped solve some of them." 

(People Need People) 

"I learned from THE HOUSE that sometimes I was similar to Carl and I 
haven't faced all my problems yet like Carl — so I learn a lot about myself." 

(From many themes) 

"I learned from the shows that other kids have problems — once everyone 
forgot my birthday — and you can learn about things you don't know and can 
relate to the kids. I learned to respect others." (From many themes.) 

Students at the case study sites were also able to demonstrate their understanding and 
appreciation for the themes when they were asked to select a favorite GALAXY book 
and to explain the reason for their selection. Many of their reasons indicate that they 
were able to make a connection between the themes and the literature. 
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Students were extremely interested in all of the themes. Two themes were especially 
popular — "People are more than they appear to be" and "It's not fair." While many 
students were reluctant to leave the "fairness" theme, many teachers jokingly expressed 
relief when they could move on to another one. Other themes dealt with values, such as 
"Be true to yourself," and some teachers were uncomfortable with them. Additionally, 
discussion and writing related to the themes occasionally led to students' painful self- 
disclosure. Not all of these students' teachers felt comfortable or adequately prepared 
to deal with such situations. 

Request for longer theme cycle 

The only criticism heard about the themes was related to the theme cycle. Teachers 
repeatedly expressed frustration with the two week theme cycle which made it difficult 
to fully utilize all aspects of the GALAXY curriculum for each theme. 

"Time constraints for each theme were too rigid. We loved the accompanying 
activities, but, felt rushed trying to complete theme. The time schedule was 
unrealistic — when you're dealing with other scheduling . . . " 

"The themes were wonderful. I very definitely need more time to fully utilize 
the materials for each theme. Two weeks isn't long enough, with all of the 
wonderful materials and ideas for each theme, I need approximately four weeks 
or longer per theme . " 

"/ would like to see more time given to each theme — I feel like I am racing to 
do activities, read literature, see shows & fax in a two week period . " 

Summary 

The GALAXY themes appealed to both teachers and students. Not only did students 
like the themes, but they also clearly understood them. Further, the themes stimulated 
the students to engage in writing and thinking and to integrate the themes into their 
conceptual frameworks. However, GALAXY would have an even greater impact if the 
theme cycle were lengthened to three or four weeks to give teachers and students more 
time on the suggested activities, to facilitate more process writing, and to create a less- 
hurried learning environment. In addition, GALAXY should consider including the 
topic of dealing with student self-disclosure in staff development discussions and 
material. 
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IV. Reading 



The Impact of GALAXY on Reading 

The GALAXY approach to reading draws from current practice and knowledge about 
effective instruction in this fundamental proficiency, by having reading become central 
to each student's continuing exploration of themes relevant to his/her own life. Within 
a GALAXY classroom, literature provides the focus for discussion, class activities, and 
writing, all thematically integrated. Reading is designed to be a satisfying and 
enjoyable activity, rather than merely a classroom exercise. 

GALAXY'S intended effects are facilitated by enhancing teachers' skills in supporting 
students in a literature-based learning environment, by increasing student motivation, 
and by increasing experience with reading for meaning. Each of these factors, to the 
extent GALAXY is successful, has the potential to produce both short and long term 
impacts on the growth of reading proficiencies. 

This chapter reports on the evidence for the effectiveness of GALAXY in changing 
reading instruction, in modifying attitudes toward the teaching and learning of reading, 
and in improving tested reading achievement during its initial fourteen-week 
implementation. 



Teacher Reports on GALAXY'S Reading Component 

Teachers at several case study sites were interviewed about their classroom practices 
and about the reactions of students during the course of the project. They also 
responded at the end of the year to a survey that included open-ended questions on the 
impact of GALAXY on their teaching and on their students. A number of observations 
arise from these sources. 

The teaching of language arts changed as a result of GALAXY. A number of teachers 
reported that GALAXY was instrumental in effecting a major change in the way they 
taught language arts, including their utilization of reading materials. Virtually every 
classroom saw a change to the use of GALAXY'S thematically based literature 
selections, as opposed to the use of basal readers or thematically disconnected pieces of 
literature. 

There also is much evidence that the thematic structure of the curriculum was 
successful in integrating the reading component of the curriculum with other learning 
activities, including writing and those activities centered on responses to the television 
programs. 
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The books selected by GALAXY for classrooms were enthusiastically received by both 
teachers and students. Interviews with teachers through the FWL case studies indicated 
unalloyed enthusiasm for the reading materials made available as a part of the 
curriculum — all of which were selected to be quality children's literature closely 
integrated with the GALAXY themes and activities. On the FWL end of year survey, 
91% of teachers gave the GALAXY literature books the highest possible rating, that of 
"Great Value." 

Teacher comments about reading on the end of year survey were enthusiastic, as the 
following samples indicate: 

"As a teacher the literature component was impressive. I used more literature 
in my lessons on a regular basis. I also felt motivated to discover other new 
titles to reflect the message of the themes." 

“I introduced children to literature that I would not ordinarily have read to 
them." 

"The literature books are by far the best selection of multicultural stories that I 
have ever read to my classroom." 

"I had wanted to implement a literature-based reading program into my 
classroom but I felt really tied to the basal." 

"The best part of GALAXY is the thematic approach. Literature selections are 
great." 

"GALAXY has ... guided me to some outstanding new children's literature." 

"GALAXY has changed the way I teach reading." 

Student Reactions to GALAXY’S Reading Component 

There are four sources of data on the reaction of students. Groups of students were 
interviewed at the case study sites near the end of the Language Arts series and asked 
to rate the GALAXY literature; an attitude survey was administered to students; 
teachers were interviewed at the case study sites throughout the program about the 
reaction of students; and the end of year survey provided another opportunity for 
teachers to conunent. 

End of year interviews with eighty GALAXY students, detailed further in Chapter VIII 
on GALAXY components, show that 84% reported that they greatly enjoyed reading the 
books, with only 16% having mixed feelings and none negative. Those who had mixed 
feelings about the literature primarily noted that they thought some of the books 
selected by their teachers were "too easy." When asked to indicate their satisfaction 
with having the GALAXY books read to them — a common practice in almost all 
classrooms — 72% gave a highly favorable rating, while 20% were more cautionary and 
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8% were unfavorable, primarily on the basis of the style and expression used by the 
teacher. 

A pre- and post-measure assessing an array of relevant student attitudes was 
administered to the GALAXY and comparison students. One scale, developed for the 
purpose of the evaluation, was a seven-item scale on attitudes toward reading. While 
this paper-and-pencil measure did not reveal significant differences between GALAXY 
and comparison students on an overall reading scale, GALAXY students showed 
significantly higher change scores on one of the individual items. GALAXY students 
showed a positive change in "liking to read in their spare time." They also tended to 
have higher positive change than the comparison students on other individual items, 
such as viewing themselves as good readers, but not enough higher to reach statistical 
significance. We must note that the results for individual scale items are somewhat 
suspect, given chance statistical variations; nevertheless, these results confirm other 
evidence that student views toward reading and their own competence in reading 
began to change as a result of GALAXY. 

Teachers reported very positive student responses to the GALAXY literature, as 
illustrated by the following examples: 

"My children were motivated to read independently; even the reluctant reader 
was taking books home and really reading them." 

"I loved the themes and the wonderful books selected to accompany them. " 

"... watching my students loving to read and write more was extremely 
rewarding." 

“The students love to hear me orally read the new GALAXY books for each 
theme which motivates t^'em to want to read the books." 

"GALAXY has not only increased my students love for reading, but it has 
also increased my love for reading." 

"I watched with pride as my children's writing and reading skills improved." 



Cognitive Measures ot Reading: Student Results on the California 
Achievement Test 

Given the start-up nature of the period under evaluation and its brief duration, 
expectations for demonstrably higher gains in reading proficiency for GALAXY 
students were not high. Nevertheless, both FWL and GALAXY'S management wanted 
to assess any such gains quantitatively and carefully. After much consideration, FWL 
decided to utilize a norm-referenced test to assess growth in reading proficiency. There 
were several reasons for this decision: such measures remain key indicators for many 
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school boards; Chapter 1 programs, one of the key target groups for GALAXY, require 
such performance measures to meet Federal and State requirements; and finally, the 
refined psychometric characteristics of these tests provide the potential for 
comparatively sensitive measures. 

The Educational Testing Service, on behalf of FWL, undertook a review of available 
national standardized tests, through its National Test Center in Atlanta, and 
recommended the California Achievement Tests, Fifth Edition, (CAT/5), published by 
CTB Macmillan /McGraw-Hill. This test is widely used throughout the country. 

FWL chose to use the Vocabulary and Reading Comprehension subtests of CAT/5, 
which together make up the Reading Total Score. These measures have been developed 
by CTB in recent years to be more reflective of whole language theory, notably by 
providing more contextual meaning for items used in the tests. 

FWL's rationale was that, if meaning-centered approaches are working well in the 
classroom. Vocabulary, as well as the ability to comprehend what is being read, would 
be positively affected, and those enhanced proficiencies would be measurable on 
commonly used standardized tests as well as on more "authentic" measures. 
Additionally, FWL employed other, less-structured methods to assess cognitive gains 
and writing proficiencies, as outlined in the discussion of the evaluation design at the 
beginning of this report. 

The CAT/5 reading subtests were administered to both GALAXY and comparison 
students by their teachers prior to tl.j initiation of GALAXY, and again in the final 
weeks of the semester. The CAT/5 uses different tests for each grade level, and FWL 
used the same form for both the pre-test and the post-test in order to cut down on the 
variation due to switching forms. 

FWL performed two different analyses on the CAT/5 data, the first using all the student 
data for matched GALAXY and comparison classrooms in the same grade at the same 
school (55 classrooms in 21 schools with Chapter 1 students) and the second using 
student data from all GALAXY and comparison classrooms that had the relevant data 
(146 classrooms in 40 schools with Chapter 1 students). The first analysis is the more 
powerful of the two since it controlled for both grade and school, which is often a 
source of variation in terms of school culture, leadership, and student characteristics. 
The second analysis was more generalizable in that it used a greater mix of classrooms 
than did the first analysis. This analysis included some third, fourth, and fifth grade 
classrooms in non-GALAXY schools in school districts that had at least one GALAXY 
school. These schools were recommended to FWL by school district administrators as 
being similar to the GALAXY school in terms of demographics and other school 
characteristics. 

FWL analyzed the scale scores from the CAT/ 5, which are units of a single, equal- 
interval scale. The same scale is applied across all levels of CAT/5 regardless of grade 
or time of year when the test is administered. The effective range of scores for each 
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grade, as one might expect, is slightly different. The obtained scale scores are based on 
the overall pattern of correct responses, rather than simply on the number of correct 
responses. Further, FWL used gain scores in analyzing the CAT/5 data because the 
evaluation focused on change over time (''gain"). 

Analysis of matched sample 

The analysis results in Table 3, based on the analysis of the 55 classroom matched 
sample of 26 GALAXY and 29 comparison classrooms, show that GALAXY students 
achieved a significant gain in scale scores over the fourteen week course of GALAXY. 
These results demonstrate a statistically significant greater gain for GALAXY students 
in twenty-one different GALAXY schools throughout the country as compared with 
comparison students in the same grade and same school, in Vocabulary, Reading 
Comprehension, and overall Reading Total. Indeed, GALAXY students gained almost 
twice as much in Reading Comprehension (90% greater gain) and nearly fifty percent 
more in Vocabulary (47% greater gain) during the tested period than did the 
comparison students in the same schools. ^ 



Table 3. Mean CAT/5 Scale Score Gains for GALAXY and Comparison Students at the 

Same Grade Level In the Same School 









aDHtSf6ncoV< 








Mean 


12.84 


8.73 


+ 4.12 


2.78 


.006 


Vocabulary 


Standard Error 


1.10 


1.03 


1.48 






Reading 


Mean 


8.11 


4.26 


+ 3.84 


2.70 


.007 


Comprehension 


Standard Error 


1.06 


1.03 


1.42 






Reading Total 


Mean 

Standard Error 


11.09 

0.88 


6.68 

0.84 


+ 4.41 

1.18 


3.73 


<.001 



A more generalizable sample of classrooms 

A more comprehensive analysis contrasts all relevant GALAXY classrooms with all 
available comparison classrooms (both those in GALAXY schools and those in non- 
GALAXY schools) in 40 schools. Again a statistically significant pattern emerges, as 
shown in Table 4, even when this larger sample of 2,826 students in 146 classrooms is 
used: the GALAXY students show a gain on Vocabulary that is 37% greater than that of 
the comparison students, and their gain on Reading Comprehension is 145% greater 
than that of the comparison students. 



I The means presented are weighted estimated means, designed to adjust for unreliable test scores, as 
discussed in the Appendix A 
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Table 4. Mean CAT/5 Scale Score Gains for GALAXY and Comparison Students in 146 

Classrooms 







GALAXY 
Gain ' 


Comparison 

Gain 


DIfferance- 

inrOaln 




P 


Voca,*' ilary 


Mean 

'Standard Error 


13.80 

1.03 


10.05 

1.24 


+ 3.76 

1.61 


2.34 


. uu 6 


Reading 

Comprehension 


Mean 

Standard Error 


7.67 

1.22 


3.13 

1.52 


+ 4.53 

1.95 


2.33 


.021 


Reading Total 


Mean 

Standard Error 


11.62 

0.97 


6.81 

1.19 


+ 4.81 

1.54 


3.13 


.002 



While slight differences appear in these two analyses, the patterns consistently show 
that GALAXY students had significantly greater gains than comparison students. 

One gauge of the performance of GALAXY students is to estimate the gain that would 
be expected of students to maintain their relative rank between testing periods (e.g., 
stay at the fiftieth percentile). An examination of the CTB norms indicates that, in order 
to hold their own, students have to gain from about 18 to 20 points in scale score from 
one year to the next. For the one semester of the GALAXY intervention, a gain of 10 
points might therefore be expected for a cross section of American children. On the 
Vocabulary subtest, the data in Table 4 show that GALAXY students overall, with their 
tested gain of more than 13 points, performed above expectation by more than 30%, 
whereas the comparison students were right on target, gaining about 10 scale score 
points. 

GALAXY gains compared to national norms 

In order to see how GALAXY and comparison students fared with regard to their 
national counterparts in the norming sample, which included a cross-section of all 
American students, rich and poor, in public and private schools, we have translated the 
Scale Score gains into Normal Curve Equivalents (NCEs) and Percentile ranks, for the 
composite Reading Total score on CAT/5, as shown in Table 5. We combined grades 
and estimated appropriate NCEs and Percentiles. 
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Table 5. Change in Percentile and NCE Ranks of GALAXY and Comparison Classrooms 

on CAT/5 Reading Total Score 





QAUOCY 


Compsti^ 


NCE Pre-Test Rcir\k 


45 


47 


Post-Test Rank 


43 


43 


Pre-Post Charige 


-2 


-4 


Perc«ntll» R«ik Pre-Test Rank 


40 


44 


Post-Test Rank 


36 


36 



As Table 5 demonstrates, students in both GALAXY and comparison classrooms 
dropped from pre- to post-test. Unfortunately, the most recent data on disadvantaged. 
Chapter 1 schools suggests it is a common pattern for such students to fall further 
behind their peers as they proceed through school. A recently published assessment of 
Chapter 1 programs by the U.S. Department of Education found a consistent drop, 
from the third to fourth grade, of three percentile points on both reading and math 
standardized test scores. The authors of that report note that these results are consistent 
with an earlier longitudinal study and show "that the gains of Chapter 1 participants 
did not match the progress of a representative sample of all students generally. 

GALAXY students, however, came much closer to holding their own in relation to their 
pj'^rs. Participants in GALAXY dropped only half as much on NCE and percentile 
measures as did their comparison class counterparts. These data support the conclusion 
that, in its first iteration, the GALAXY Language Arts curriculum served to ameliorate 
the reduced rate of growth of cognitive proficiencies that characterizes America's 
disadvantaged children. 

The NCE and percentile data, however, should not be used as more than a rough 
indicator. The slight drop for GALAXY students is somewhat inconsistent with the 
earlier demonstration that students in GALAXY classrooms exceeded performance 
expectations for the tested period. Since the NCE and percentile data are based on 
combined norming populations for the different grades, and since typically test-retest 
does not occur in so short a period, the NCE and percentile data may be somewhat 
misleading if taken in terms of absolutes. Only a longitudinal study of GALAXY 
students over a more extended period of time than four months would settle the issue 
definitively. 



^Reinventing Chapter 1: The Current Chapter 1 Program and New Directions. U.S. Department of Education, 
1993. 
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Does the effect of GALAXY depend on ethnicity, gender, or other student 
characteristics? 



Because GALAXY is designed to meet the needs of the increasingly multicultural 
populations of our schools, the evaluation team deemed it important to determine 
whether it served all those populations successfully during the demonstration phase. 
Appropriate statistical analyses were carried out to test whether the gains by the 
GALAXY students depended on grade level, gender, ethnicity. Chapter 1 status, home 
language other than English, and special education status. All of these analyses showed 
that the gains did not differ significantly by any of these categories. Statistically 
speaking, there were no significant main effects for these variables nor were there any 
interactions with the treatment that were statistically significant. 

The demonstrated advantage of GALAXY is not the result of differential impacts among 
ethnic groups, whether African-American, Hispanic, Asian American, Native American, 
or other. Neither does GALAXY's advantage rest upon differential impact on girls and 
boys. GALAXY students had pre-test scores that were related to what grade they were 
in (e.g., 4th graders had higher initial scores than 3rd graders), but the magnitude of 
their gains was not related to what grade they were in. That is, GALAXY effects were 
similar in third, fourth, and fifth grades. A further analysis showed that the gains 
shown by GALAXY students were not dependent upon status as a Chapter 1 recipient, 
whether the language spoken at home was English, or Special Education status. It is 
therefore appropriate, in the rest of this report, to discuss the impact of GALAXY 
without regard to these special populations. 



The range of performance among classrooms 

As with all studies of classroom performance, we found great variation among 
classrooms in both their initial scores and in their gains. Schools differ, children differ 
in their backgrounds, and perhaps most importantly teachers differ. 
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Rgure 1. Mean Gain on Vocabulary for 93 GALAXY Classrooms 
Vocabulary Gain Scoie 




GALAXY Classrooms 



GALAXY classrooms varied widely in their gain scores, as did comparison classrooms. 
Graphical representations of the rank order of the two groups of classrooms in terms of 
the Vocabulary subtest are displayed separately in Figures 1 and 2 which show the 
mean gains and standard err'_ .s for GALAXY and comparison classrooms arranged 
from highest to lowest. 

Note from Figure 1 that some GALAXY classrooms had a mean gain of nearly 40 points 
at the high end, while only five of the ninety-three classrooms (5%) failed to post a 
Vocabulary gain. 
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Figure 2. Mean Gain on Vocabulary for 53 Comparison Classrooms 




Figure 2 shows that a few exceptional comparison classrooms gained, on average, more 
than 30 points, but seven of the fifty-three (13%) failed to make a gain. Indeed, the 
curve that one would get from connecting the means with a line seems to be similarly 
shaped to that for the GALAXY classrooms. 
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Rgure 3. Mean Vocabulary Gain Score by Classroom 




However, when the two curves for the ninety-three GALAXY and fifty-three 
comparison classrooms are overlaid, as in Figure 3, it becomes apparent that the entire 
distribution of GALAXY classrooms is shifted toward better performance. That is, 
GALAXY classroom at the eightieth percentile outperformed comparison classrooms at 
the same percentile; the same is true at the twentieth percentile. Furthermore, as Figure 
3 illustrates, 17% of GALAXY classrooms showed gains in the Vocabulary subtest of 
twenty points or more, twice as great as the expected 10 point gain. Only 9% of 
comparison classrooms made similar gains. 
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Figure 4. Mean Reading Comprehension Gain Score by Classroom 



Reading Comprehension Gain Score 




Figure 4 shows the similar but less dramatic results for Reading Comprehension. It 
indicates that, for the vast majority of students in classrooms in the middle of the 
distribution, GALAXY classrooms had higher Reading Comprehension gains than 
comparison classrooms. 
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Figure 5. Mean Reading Total Gain Score by Classroom 
Reading Total Gain Score 




Percentile Rank of Classroom 
Classroom GALAXY Comparison 



Figure 5 represents the mean overall Reading Totals for GALAXY and comparison 
classrooms. Since the Reading Total is based on the combined Vocabulary and Reading 
Comprehension scores, it is not surprising that the evidence indicates that GALAXY 
students outperformed their counterparts. 



An examination of these figures shows that, for classrooms that performed at a variety 
of levels throughout the range, greater gains are to be found consistently for the 
GALAXY classrooms. 
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What characterizes more and less successful GALAXY classrooms? 



The CAT /5 test revealed a substantial range of performance in GALAXY classrooms, 
and this suggests the utility of analyses to identify some of the sources of those 
differences. To the extent that they reflect factors that can be modified by GALAXY'S 
management, they may represent a potential for increasing the effectiveness of 
GALAXY in the future. 

The evaluation gathered a good deal of information provided by teachers, principals, 
and students about the characteristics of participating classrooms. The end of year 
teacher survey (see Appendix K) contained questions addressing such matters as the 
amount of time spent using the various GALAXY technologies, the amount of time they 
were fully operational, and the perceived educational value of each of the instructional 
components. A pre/post survey on instructional practices and beliefs provided further 
information. Finally, detailed teacher logs of instructional time gave insight into the 
amount of time spent teaching each of the GALAXY themes and the specific activities 
that were used. 

FWL has analyzed this information base to ascertain whether certain characteristics 
define either the best-performing classes of GALAXY learners or the poorest 
performers. The analysis was done for the entire population of GALAXY classrooms on 
which data was available.^ 

The relationship was analyzed in two ways; through correlations with Vocabulary and 
Reading Comprehension gain scores and through an analysis of groups of classrooms 
divided into categories on predictive variables, to determine the effect on average 
classroom cognitive performance. 

As one can see from Table 6, the analysis showed small, but statistically significant, 
positive correlations of Vocabulary gains with several measures including using 
GALAXY as a replacement for the traditional language arts, spending time on 
GALAXY, and using the fax machine. Interestingly, none of the three factors in Table 6 
had a significant correlation with gains in Reading Comprehension. As later analyses in 
this chapter show, however, classrooms at the high end on these variables did have 
greater Vocabulary gains. 



^Teachers were not wholly consistent in providing information on their classroom practices; therefore, the 
number of teacher responses varies somewhat from question to question. 
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Table 6. Vocabulary Gain and GALAXY Language Arts Teaching Intensity 



Variable 


Correlation With Vocabulary 
Gain (Slgnificanca) 


Correlation With Reading 
Comprehension Gain 
(Significance) 


Teacher report: GALAXY used as a 
replacement rather than as supplement 


.33 


-.03 


for regular Language .Arts teaching 


(p=.002) 


(ns) 


(N=89) 






Time spent teaching GALAXY per 


; .38 


: -.12 


theme (N=59) 


! (p=.002) 


(ns) 


Frequency of use of the fax machine 


.35 


, -.10 


(N=74) 


1 (p=.002) 


• (ns) 



(ns)=not Significant 



The variables that have an impact on performance, as detailed in Table 6, point to an 
intensity of use in the GALAXY classrooms. If we combine these variables, the data 
suggest that the most effective classrooms; embraced GALAXY more fully, tending to 
use its rich array of activities as a complete curriculum; spent more time in teaching 
GALAXY; and, very interestingly, made much greater use of GALAXY'S primary 
interactive instructional component, the fax machine. 

Thie three tables following show the effect of these three factors, when categorized, on 
average Vocabulary and Reading Comprehension Q,*ms in these classrooms. 



Table 7. Mean Scale Score Gains for GALAXY Supplement/Replacement Classrooms 





Ua«dQAtAXYMa 
Language Arte SuppiMiMnt 
(79 teachera) 


UMd GALAXY asa 
Language Ms 
Reiilacement 
(10 teachers) 


t 


P 


Vocabulary Scale Score Gain 


12.5 


21.1 


3.25 


.002 


Reading Comprehension 
Sccde Score Gain 


7.5 


6.4 


-0.26 


.80 



As Table 7 indicates, the few teachers who reported using GALAXY as a replacement 
for their traditional language arts teaching produced notably greater improvement on 
Vocabulary scores (p=.002). This could be interpreted as an affirmation that an effective 
meaning-based approach, such as that made possible by GALAXY, can be particularly 
effective in ervhancing Vocabulary achievement, an important cognitive measure. At the 
same time, we must acknowledge the possibility that these ten teachers differed in other 
ways, as well, that may have made them more effective teachers. For example, they 
may have been more committed to the GALAXY philosophy than other teachers, or 
they may have been real trailblazers for their schools. 
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VVe should note that the GALAXY Institute staff explicitly encouraged teachers and 
principals to think of GALAXY language arts as a supplement, not as a replacement, for 
their ordinary instruction. It would appear that these ten teachers, however, did use it 
as their only Language Arts instruction. 

Time spent teaching GALAXY 

The evaluation ascertained the time spent teaching GALAXY from logs submitted by 
teachers as they worked through the seven GALAXY themes. The performance of 
students was examined for teachers who spent particularly large, particularly small, 
and average amounts of time teaching the GALAXY curriculum. Table 8 contains the 
results of those computations. 



Table 8. Mean Scale Score Gains by Time Teaching GALAXY 





Lowest 
25 percent 
tlnMKm 
OALAXY 


Middle 

50 percent time 
on GALAXY 


Highest 
25 percent 
tints on 
GALAXY 


Vocabulary Scale Score Gain 11.1 


15.1 


20.4 


Reading Comprehension Scale t g ^ 

Score Gain | 


7.8 


12.3 



The 25% of teachers who spent the most time teaching GALAXY produced substantially 
larger gains in both Vocabulary and Reading Comprehension. For the Vocabulary 
gains, there were statistically significant differences in gain between the highest and 
lowest quartile, while for Reading Comprehension the differences approached 
significance. 



Table 9. Mean Scale Score Gains by Frequency of Fax Machine Use 





OccMionelly 
or not St ait 
(18 

toacbera) 


Once per 
(U 

teacheim) 


28Hmee 

perweek 

(49 

teachers) 


Vocabulary Scale Score Gain 


8.6 


10.6 


15.8 


Reading Comprehension 
Scale Score Gain 


3.4 


7.4 


9.1 



Similarly, the data in Table 9 appear to offer confirmation of the utility of GALAXY'S 
unique interactive component, the classroom use of a fax machine, particularly when 
looking at the extremes of fax usage. Those teachers who reported using the fax two or 
more times a week — an expected utilization for a GALAXY classroom — had classrooms 
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that produced standardized test score gains almost twice as high on both Vocabulary 
and Reading Comprehension as those who used the fax only occasionally or not at all. 

The interpretation of this result will be further discussed in Chapter 8 on the overall 
importance of interactivity. It would appear that the effectiveness of GALAXY in 
improving reading proficiencies is enhanced by classroom utilization of the fax. 



Discussion of the “GALAXY Intensity” factors 

Throughout this chapter, we have noted the greater impact on Vocabulary than on 
Reading Comprehension of factors related to the intensity of students' GALAXY 
experience. In contrast. Tables 3 and 4, at the beginning of this chapter, show that 
overall there were quite large Reading Comprehension differences favoring GALAXY 
relative to comparison students. In other words, the Reading Comprehension gains 
were less dependent on the reported degree of teacher commitment to the GALAXY 
curriculum. 

One speculative explanation for what we observed is that almost all of the GALAXY 
curriculum stimulates the growth of meaningful understanding, the proficiency 
measured by Reading Comprehension. Reading or hearing the teacher read the 
GALAXY literature, discussing that literature in small groups and in class, and doing 
related activities could be expected to contribute gains in comprehension proficiency. 

The relative gain in vocabulary proficiency, on the other hand, may well be more 
dependent on the amount of exposure to certain specific learning experiences. Whole 
language research suggests that the more one reads and the more one writes, the greater 
one's mastery of vocabulary. It is thus reasonable, at least, to believe that in those 
classrooms where more time was spent on GALAXY activities, and where the greater 
use of the fax promoted more student writing, greater vocabulary mastery would result. 



Comments on GALAXY’S Reading Strategies 

The case study observations suggested that students spent a relatively small amount of 
instructional time reading on their own, while they spent a good deal of time hearing 
the literature read to them, discussing ideas from the literature with other children in 
cooperative groups, and writing. The CAT/5 data suggest that this kind of experience 
transferred to higher scores on tests that demand reading for meaning and 
comprehension of written vocabulary, an affirmation of GALAXY'S effectiveness in 
delivering meaning-based instruction. 

At the same time, one of the most consistent recommendations coming from teachers in 
the case study schools observed by FWL was that a larger number of copies of the 
GALAXY literature be made available, to permit increased reading both in class and 
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outside of class. Having set the stage so well for the enjoyment and understanding of 
literature, it would seem appropriate to encourage schools to nurture that interest to the 
maximum possible extent, by making available as many of the excellent books selected 
by GALAXY as possible. 



Summary 

GALAXY'S literature-based reading strategies proved extraordinarily popular with both 
teachers and students. Many teachers modified the way they taught language arts, 
including its reading component. Students were very enthusiastic about the literature 
that they read and had read to them. They also showed promising changes in their 
attitudes toward reading, reporting a greater propensity to read books on their own 
time and to feel that they themselves were good readers. 

The analyses of CAT/5 test scores show significant advantages to students who have 
participated in GALAXY, as contrasted with those in comparison classrooms. Based 
upon the more generalizable sample of 146 classrooms, in Vocabulary, the average 
growth of student proficiency was 37% greater than for comparison students; in 
Reading Comprehension, growth was one and a half times as great. The result was that 
experience with GALAXY ameliorated the relative decline in language proficiency 
scores that is characteristic of disadvantaged students and that was found in the 
comparison classrooms. 

There was, however, wide classroom-to-classroom variation, both in achievement and 
in implementation of GALAXY. An analysis shows that the classrooms in which 
GALAXY was taught for the greater time had appreciably higher scores, particularly on 
Vocabulary. For a subset of classrooms where teachers chose to substitute GALAXY for 
their traditional language arts teaching. Vocabulary gain soared to twice that of 
comparison classrooms. And for classrooms that used the fax capability of GALAXY as 
expected, two to five times a week, higher Vocabulary gains resulted. The impact of 
these factors was more limited, although in the same direction, for Reading 
Comprehension. Those gains would be sufficient, in the case of vocabulary 
development at least, to raise the rank of students relative to their more advantaged 
counterparts throughout the nation. 

These findings suggest that the full potential of GALAXY has not yet been reached. 
Because of an uneven implementation in this demonstration phase and technical 
problems that prevented the fax from being effectively used for long periods in a 
number of classrooms, the utilization of GALAXY was less than optimal in many 
classrooms. The data suggest that, when GALAXY is more fully implemented, even 
more significant cognitive gains may be achieved. 
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V. Writing 

"GALAXY encourages me to have my students think, think, think 
and write, write, write!" 



— comment from GALAXY teacher 



Introduction 

Student writing receives a strong emphasis in the GALAXY Language Arts curriculum 
for grades three through five. FWL researchers gathered observations related to 
writing during the GALAXY Language Arts cycle from interviews with teachers and 
students conducted during case study site visits. This chapter begins with teachers' 
views on student attitudes towards writing. Next, students' own words will be used to 
illustrate their views on writing. Additionally, the end of year teacher survey generated 
comments on writing. This chapter then discusses findings from two types of writing 
measures designed to demonstrate students' ability to write persuasively, the pre/post 
writing prompts and four embedded performance assessments. 



Teacher Reflections on the GALAXY Writing Experience 

The emphasis on writing in the GALAXY Language Arts curriculum and classroom 
activities has had a positive impact on students, according to their teachers. This 
finding is based on evidence from the end of year teacher survey and from interviews 
with teachers at the case study sites. 



Survey Findings 

Data and comments from the end of year teacher survey corroborate the finding from 
the case study schools that teachers have observed positive changes in their students' 
writing behaviors. In discussions with FWL researchers, a number of teachers at the 
case study sites reported that their students were writing more emd writing better as a 
result of GALAXY. Their observations are supported by the teachers' responses on the 
end of year teacher survey; over one hundred teachers responded to this survey. As 
Table 10 shows, all the responding teachers thought that GALAXY did "very well" or 
"somewhat well" in helping students to brainstorm their writing ideas, and over 95% 
similarly rated GALAXY in terms of encouraging students to write more frequently. 
Writing for a real audience also scored well with the teachers, with all but one of them 
rating GALAXY as doing "very well" or "somewhat well" on this dimension. These 
ratings which reflect positive changes in students' writing behaviors may well be 
related to GALAXY'S irmovative use of the fax machine to communicate with the 
broadcast producers, THE MAX, and partner schools. Students' use of the fax machine 
will be discussed in greater detail in Chapter VII on interactivity. 
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Table 10. Teacher Rating of GALAKY Influence on Writing Behaviors 

(Percent of 102 Teachers) 



How well does GALAXY help students to 


Very Wall 


Somewhat i Not Very 
{ WeU i Well 


Not At All j 


Brainstorm to generate ideas for writing 


77.5 


22.5 


0.0 


0.0 


Write more frequently 


82.4 


14.7 


1.0 


2.0 


Write for "real" audiences 


76,2 


22.8 


1.0 


0.0 



In addition to rating writing behaviors as GALAXY outcomes on the end of year teacher 
survey, teachers were asked to give their impressions of whether various student 
attitudes and outcomes were true for most, some, a few, or not true for any of their 
students. To provide some context for the writing and reading outcomes, it is useful to 
know that the highest rating of any outcome was agreement by 92.3% of the teachers 
that it was true for most or some of their students. Given this context, the writing 
dimensions fared extremely well as Table 11 indicates. Overall, the highest rated item 
was agreement by teachers that some or most of their students were more willing to 
undertake writing assignments. And a majority of teachers (90%) agreed that most or 
some of their students were more willing to write on their own as a result of GALAXY. 
Perhaps the best news for GALAXY in this area is that 91% of the teachers reported that 
most or some of their students had developed a better attitude towards reading and 
writing after exposure to the GALAXY language arts program. Eighty-one percent 
thought that most or some of their students were reading more. 



Table 11. Teacher Rating of GALAXY and Writing Assistance (Percent) 



Since QAIAXY began, nqr 






sjjruoflfqr^ 




Are more willing to undertake writing assignments 


54.8 


37.5 


5.8 


1.9 


Are more willing to write on their own 


51.0 


38.5 


7.7 


2.9 


Have a better attitude about reading and writing 


59.0 


32.0 


9.0 


0.0 


Read more 


51.0 


29.8 


17.3 


1.9 



When asked to choose the three most important results of GALAXY for their students in 
the end of year survey from among these and nine other items, more teachers selected 
aspects of writing than any other topic. The top two vote-getters among the thirteen 
choices both had to do with writing; 40% of teachers selected producing better attitudes 
among the students towards reading and writing as a significant outcome of GALAXY, 
and 38% of the teachers selected an improved willingness of students to do writing 
assignments as one of their three most important outcomes. (Complete results of the 
end of year teacher survey appear in Appendix K.) 
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Process Writing 



The GALAXY Language Arts curriculum emphasizes the various aspects of process 
writing (e.g., pre-writing, drafts, revising for meaning, editing, and final production). 
Both case study data and survey data confirm that GALAXY students did not engage in 
much process writing during the course of the fourteen-week GALAXY cycle. 

Although many of the suggested GALAXY activities are pre-writing tasks (e.g., 
mapping, webs, Venn diagrams, and note taking), and students were often observed 
engaging in various pre-writing exercises, an analysis of student written work kept in 
the portfolios of fifteen of the GALAXY classrooms at the case study sites indicates that 
a relatively low percentage of their total GALAXY writing work was of this type 
(15.9%). In three of these classrooms, where pre-writing was the predominant writing 
task found in portfolios, 39.0% of the student work was of this type. A further analysis 
of the extent to which pre-writing work was linked to subsequent writing products 
(e.g., drafts, edited work, final products) revealed very few such linkages. In addition, 
there was little evidence that students had opportunities for editing and especially for 
revising their work. 



Table 12. Teacher Rating of GALAXY Influence on Writing Behaviors 

(Percent of 102 Teachers) 



How well does GALAXY help students to j 


1 

_Vafy:Wj^. 


j^JSonnmwhat .. 




Revise wntmg for meaning | 


37.6 i 54.5 7.9 



The end of year teacher survey confirms this view from the case studies that very little 
fully-developed process writing was taking place. In contrast to other aspects of 
GALAXY rated more highly by a majority of teachers (as we saw above), only 38% of 
the teachers thought that GALAXY did very well at assisting students to revise for 
meaning (although an additional 55% of the respondents thought GALAXY did this 
task somewhat well). 

The hectic pace of the GALAXY language arts cycle may explain the lack of process 
writing. As one teacher said, 

"Because there was always a time constraint, it was difficult to find time for 
revising. For example, we'd watch the 'A' show on Tuesday and anything 
being faxed to THE HOUSE had to be ready by Wednesday morning. Not 
enough time to write, think about, edit and rewrite." 

In order to allow for more process writing in the future, GALAXY should consider 
allowing more time betv^een the A and B shows and a bigger window for faxing to The 
House. 
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Teacher Open-Ended Comments 

At the case study schools, almost all teachers report that in the GALAXY language arts 
program their students are writing more and with much more fluency than they 
thought possible. They report this to be true for students of all ability levels. In fact, 
there are dramatic stories about individual students who have begun writing "in 
volumes" since they have been part of GALAXY. Teachers recognize that GALAXY has 
equipped them with important tools tha*: have been instrumental in increasing students' 
writing: the activities guide; the highly popular broadcasts, which present their students 
v/ith real problems about which they want to express an opinion; and fax machines, 
which encourage students to write to a real audience. 

In general, GALAXY teachers were aware that how they taught writing had changed, 
that students were more willing to write, and that students' writing had improved as a 
result of their collective GALAXY experience. Here are some comments that teachers 
made when they reflected on what GALAXY had meant for them and their students: 

Changes in teaching writing 

"GALAXY has brought about a change in my teaching of writing. I allow my 
students to take chances and experiment with language. Daily opportunities 
are provided now for writing for various purposes. I realize that writers can 
move back and forth between steps rather than through a series of steps with 
correct spelling and punctuation." 

"I have finally realized that when you make their writing assignments more 
personal, the students write more, write better, and they 'think' about what 
they're writing." 

"[GALAXY] has helped me see how to teach xvriting much more effectively." 

"I have stressed writing for ideas more & then build into the grammar." 

Changes in students’ willingness to write 

"Using the GALAXY program has increased my students' learning ability. 

They are eager to write their thoughts down. It encourages the students to 
brainstorm their ideas." 

"It has also motivated the students to want to write more — they love to write 
to The House, etc." 

"The children automatically, without being told to write, wanted to write." 

"Love the quick writes! Many times when the show is over they just start 
writing about what happened and then start preparing faxes. Many report 
on books they're reading & link it with a GALAXY Theme." 

38 
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Improvement in student writing 

"[ noticed an improvement in writing — more so than in past years." 

"The zuriting skills of my students has surpassed my wildest dreams. We've 
all benefited from the GALAXY Programl" 

Ability levels of students 

“We are ability grouped. We observed that the low achievers profited more 
than the high achievers. We felt that the high achievers would have progressed 
regardless. " 

"Since GALAXY I have seen a tremendous improvement in my LD Resource 
students' writing. The more we write, the better their writings get." 



Student Attitudes Toward Writing 

Although a student attitude survey failed to uncover significant changes in student 
attitudes towards writing, small-group interviews with students at the case study 
schools did show that students recognized many of the attributes of GALAXY writing. 

One of the scales of the student attitude survey was designed to measure attitudes 
toward writing. The seven writing items in the writing scale sought to probe student 
attitudes toward themselves as writers, the amount of writing they did at school, the 
importance of learning to write well, and revising their writing. Writing items were 
phrased like the following item: "Some kids wish they could do more writing in school 
But Other kids wish they could spend less time writing in school." (See Appendix B for 
the full text of the student attitude survey.) Students were then asked to choose which 
of the statements applied more to them and whether it was really true for her/him or 
sort of true. However, neither the change in this scale overall nor the change in any of 
the items in the scale was significantly different for GALAXY students as compared 
with comparison students. 

Students in all the case study sites appeared to be aware that their GALAXY writing 
differed from prior language arts writing experiences. When asked to describe the kind 
of writing assignments they did in the previous school year and the kind they do in 
GALAXY, they noted very different types. Previous writing assignments were 
frequently described in terms of book reports, writing about newspaper articles, writing 
stories using spelling words, doing reports on famous people, doing reading skills 
workbooks, keeping journals, writing sentences, and completing dot-to-dot coloring 
sheets. When asked to tell about the kinds of writing they were doing in GALAXY, 
students across all sites were able to provide examples of writing more, writing for an 
authentic audience, and writing about important topics. They offered the following 
examples: 
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Writing More 

"now we write a lot — we write poems and stories and letters to the kids at 
THE House" 

"writing a lot of letters" 

"we write much more — letters, faxes, pre-writing, and brainstorming" 

Writing for an Authentic Audience 

"writing different types of documents including faxes, raps, birthday cards, 
and letters" 

"letters to Gloria, Cliff, Andrea and Sue" (note: these were performance 
assessment letters written to the FWL GALAXY evaluation team) 

"we get to tell our special ideas to other people around the United States. " 

Writing About Important Topics 

"writing about important topics and other neat stuff" 

"writing about problems and solving them" 

"making predictions" 

"writing solutions to problems, giving you a better opportunity to learn about 
real life" 

"writing about helpers, heroes, and people who need people" 

"writing about the shows" 

"we xvrite about our opinions" 

"advantages and disadvantages charts" 

"zvriting letters about problems like Nessie" (note: one of the writing prompts) 



Measures of Student Writing Ability 

Two different types of writing assessments were developed by researchers from the 
Educational Testing Service (ETS), in cohaboration with researchers at Far West 
Laboratory, and given to GALAXY students: pre/post writing prompts, administered 
in January and May 1993, and embedded performance assessments, administered 
during weeks 4, 6, 12, and 14 of the GALAXY language arts cycle. 
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Pre/Post Writing Prompts 



Two comparable writing prompts were used in the GALAXY Language Arts 
evaluation. In consultation with staff at the GALAXY Institute, FWL and ETS chose 
"persua.si.ve" writing prompts as being closest to the GALAXY curriculum objective of 
facilitating the studentc' ability to take a stand and support it with evidence. As 
generalized writing tasks not linked to the GALAXY curriculum, these two prompts 
were considered suitable for and administered to both GALAXY and comparison 
students before and after GALAXY. 

For the pre-test writing prompt, the FWL researchers used the National Asses‘=;ment of 
Educational Progress (NAEP) "persuasive" writing prompt for grade 4 for the pre-test. 
A comparable writing prompt, developed by tFie FWL team, was used for the post-test. 
ETS conducted a comparability study of the two prompts to ensure that the tasks were 
similar. They administered the two tasks in counterbalanced order to two classes in the 
same grade approximately one week apart. Their results showed that there were no 
significant differences between the two writing prompts. 

The scoring rubric developed by NAEP for scoring the writing prompt was used to 
score both pre- and post-tests. A six-point scale (1 to 6) was used to differentiate among 
the writing levels ej^ibited by the students. (See Appendices C, D, and J for the text of 
the writing prompts and the rubrics.) Both pre- and post-GALAXY administrations 
were scored in a single sitting by ETS in order to remove as much variation as possible 
from the scoring. Ten percent of the papers were scored by two readers and re-scored 
by the chief reader if their two scores differed by more than one point. Reliability 
among the readers was assessed to be high. 

The writing prompts proved to be a powerful enough measure to detect writing gains 
among the students. To help us detennine the educational significance of these gains, 
consider the different initial scores (score on the pre-test writing prompt) for students in 
the three grades as indicated in Table 13: 



Table 13 . Writing Prompt Scores by Grade for GALAXY and Comparison Students 




Table 13 shows that, for all students combined, the difference in initial scores between 
grades 3 and 4 and between grades 4 and 5 is about .3 on the pre-test W riting Prompt 
(WPl). We would then expect to see a gain between .3/3 (since January to May is 
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about 1/3 of calendar year) and .3/2 ^since January to May is about 1/2 of an academic 
year). We therefore expected to see a gain of between .10 and .15. What we found was 
that GALAXY students had a mean gain of .125 and comparison students had a mean 
gain of .136. Both of these gains ire statistically significant at the .01 level, indicating 
that a real gain in writing performance has been measured. However, as one might 
imagine from the similarity of the gains of the two groups, there was no statistically 
significant difference in gain between GALAXY and comparison students. Moreover, 
differences in initial scores did not account for the slight differences in gain between the 
two groups since they had initial means on WPl that were not statistically different: 
GALAXY students overall had a mean of 2.90 and comparison students scored 2.86. 

There were some differential gains by grade, as the Table 13 indicates. The lower mean 
gain for the fifth graders may be partially attrih’itable to a ceiling effect of having a scale 
that ranged only from 1 to 6 and using the same scale for all three grades. That is, fifth 
graders started higher and did not have as far to go. As Figure 6 shows, the GALAXY 
students did gain more than comparison students in third and fourth grades. Indeed, 
they seem to be rapidly making up ground: the GALAXY third graders have WP2 
scores that are higher than the WPl scores of the fourth graders, and the fourth graders 
have WP2 scores that are not too different from the WPl scores of the fifth graders. The 
fifth graders, however, had disappointing results, as Figure 6 shows. 

Figure 6. Writing Prompt Gains: GALAXY and Comparison Students 
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At the classroom level, the results were quite similar. Figure 7 shows that when both 
GALAXY and comparison classrooms are ranked from highest to lowest, GALAXY 
classrooms usually, but not always, display greater writing gains. However, there are 
enough occurrences of similar scores to prevent the overall pattern from being 
statistically significant. 



Figure 7. Mean Writing Prompt Gains for GALAXY and Comparison Classrooms 




In summary, the good news is that GALAXY students made writing gains of the 
magnitude that were expected, but then so did the comparison students. The data show 
that GALAXY students did not perform significantly better than comparison students 
on this "persuasive" writing test. 

There are a couple of possible interpretations for this, one having to do with the scoring 
rubric and another having to do with the restricted scale used. The NAEP scoring 
rubric for the first writing prompt and the nearly identical one for the second writing 
prompt were designed to measure "persuasion through articulation and support of a 
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position." Further, the scoring rationale "requires respondents to take a position" about 
the issue in question (whether to allow a spaceship to return to its planet for WPl and 
whether to capture the Loch Ness monster for scientific purposes for WP2) and "to 
provide reasons to support their position." The writing tasks thus tested increasing 
ability to take a stand, that is, to marshal evidence in support of an argument; this is one 
of the key objectives of the GALAXY language arts curriculum. However, the 
experience of the GALAXY students with "taking a stand" in writing may have been 
minimal due to the relatively short time allotted to each theme (2 weeks). Classrooms 
tended to discuss issues related to the themes, rather than to write about them. 

It is quite possible that GALAXY students' writing changed in other ways than in their 
ability to marshal evidence. For example, the students may have learned to write more 
easily or to write more in less time or they may have become more willing and 
confident to undertake writing tasks. 

The writing prompts may not have been sufficiently sensitive to pick up small changes. 
It is possible, for example, that changes in GALAXY students' ability to take a stand 
were not sufficiently l<-rge to be picked up on a 6-point scale that spans such a broad 
ability range. 



Embedded writing performance assessments 

In consultation with FWL and GALAXY, ETS developed foiu: performance assessments 
based on activities in the GALAXY teacher's guide. The four assessments were 
designed as H 'o pairs: the first and last performance assessments took place at the end 
of themes 2 and 7 (PAl and PA4), and they were designed to allow the maximum time 
between assessments in order to pick up a gain. The middle two took place at the end 
of themes 3 and 6 (PA2 and PA3). Since the assessments were integrally linked to the 
GALAXY curriculum, there were administered only to GALAXY students and not to 
comparison groups. 

The contexts for the writing tasks varied somewhat. PAl and PA4 referenced the 
GALAXY broadcasts as the framework for the writin,*; tasks. PA2 and PA3 both used a 
short written real-life story as the context for the writing assessment. Because PA2 and 
PA3 could be done by students not familiar with GALAXY, ETS used them in a 
comparability study to ensure that the two tasks were similar. To test comparability, 
ETS staff administered the two assessments in counterbalanced order to two classes in 
the same grade approximately one week apart. The students received no instruction 
directed to the tasl^ during the intervening week. The results from this study showed 
that there were no significant differences between the two performance assessments. 

Rubrics for scoring the two pairs of performance assessments were developed by ETS 
and FWL to reflect increased mastery of "taking a stand." To the extent possible, facility 
with grammar, spelling, and punctuation were not included in the score. All 
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performance assessments for grades 3-5 were scored by ETS using the same 5-point 
scale (1 to 5).“^ 



Table 14. Mean Scores by Grade for Two Pairs of Performance Assessments 



Grade 


Mean Score 
on PAl* " 


PAl and PA4 
Mel^S££rS; 


|:Mea5i'Galn§ 


.^Mean Score? 
a^bn.PAZ^l 


PA2 and PA3 
^eah'^Soore^ 


I^Mean Galn^ 


3 


2.29 


2.48 


•19 


2.61 


2.72 


.11 


4 


2.66 


2.78 


.12 


2.86 


2.90 


.03 


5 


2.80 


2.77 


-.03 


2.87 


2.88 


.01 


Overall 


2.61 


2.68 


.07 


2.80 


2.85 


.05 



Although both pairs of assessments showed gains, the overall gains for the combined 
grades were small and not statistically significant (.07 for the mean gain between PAl 
and PA4 (p=.ll) and .05 for the gain between PA2 and PA3 (p=.22)). We expected to see 
less growth between PA2 and PAS because the two latter assessments were 
administered closer in time than the other pair: Only six weeks separated the 
administrations of PA2 and PA3, whereas there were ten weeks between PAl and PA4. 
The results confirm this expectation of less growth. 

Although there was no statistically significant relationship between grade and initial 
score for either pair, the trend of the initial scores was in the right direction, starting 
somewhat higher as the grade increased in both pairs. As Table 14 shows, the average 
score for fifth graders was higher than for fourth graders, which in turn was higher than 
for third graders on both PAl and PA2. 

It is possible to do a "growth" analysis similar to that done for the writing prompts in 
order to get some idea of "expected gain." Looking at the scores for PAl in Table 14 
above, we can see that the difference between grades 3 and 4 is .37 and the difference 
between grades 4 and 5 is .14. Similarly, looking at the scores for PA2, the difference 
between grades 3 and 4 is .25 and between grades 4 and 5 is .01. Because the differences 
between the grades have such a range, it makes sense to restrict our discussion to the 
gain we might expect for third and fourth graders. 

Viewing the ten weeks between PAl and PA4 as 19.2% of a year and 25% of a school 
year, we might expect third graders to have gains of between .071 and .093 (19.2% of .37 
and 25% of .37, respectively) on PAl. However, the mean gain of .19 was higher than 
that. Similarly, for PA2 and PA3, separated only by six weeks, we might expect third 



4 In addition, a score of 0 was assigned to those who failed to respond to the task (e.g., they only wrote 
their names). 
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graders to gain between .029 and .038 (11.5% and 15% cf .25) on the basis of the 
difference between third and fourth grades on PA2; they actually gained .11. The 
comparable figures for fourth graders for PAl and PA4 are gains between .027 and .035 
as compared with a measured gain of .12, and, for PA2 and PA3, gains between .001 and 
.002 (measured gain of .01). By this crude measure of what one might expect, the 
GALAXY students in both third and fourth grades exceeded expectations of gain on 
both sets of performance assessments. While these numbers are only suggestive, they 
do indicate that, with a larger sample size and more power to detect small changes, the 
performance assessment pairs may well show a writing gain. 



Figure 8. Performance Assessment Gains During GALAXY Language Arts 
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When one analyzes the performance tasks as repeated measures of the same skill 
("taking a stand"), then a pattern emerges, as Figure 8 shows. The performance task 
scores for the 526 GALAXY students who have scores for all four performance 
assessments show statistically significant gains on the first three embedded 
performance assessments (PA1-PA3) when they are considered together. However, 
there is no significant gain when all four performance assessments are included because 
the scores on PA4 are low, as Figure 8 shows. We know that the fourth performance 
task was taken under different circumstances than the other three. This last task (PA4) 
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Figure 8 also shows that the pattern of gains is not the same for all the grades. The 
gains for third and fourth graders show a similar trend, but the gains for fifth graders 
appear to somewhat flatten out, possibly reflecting a ceiling effect from the scoring 
rubric. 

In summary, the statistical results from the performance assessments are somewhat 
mixed. When viewed as a series, the performance assessments reveal statistically 
significant gains among GALAXY students when the fourth performance task is 
excluded. Yet when viewed as two pairs of assessments, the gains are not statistically 
significant. 

Several factors, similar to those for the writing prompts, offer partial explanations for 
the lack of significance of the paired-task analysis. <^e factor relates to a poor rate of 
return that diminished the sample size and adversely affected the probability that a 
statistically significant difference could be found. Many teachers did not return both 
halves of an assessment pair for one reason or another,^ and a number of schook closed 
for the year before reaching theme seven and its accompanying PA4. Another factor 
that may account somewhat for the lack of significance k the relatively short time frame 
between the two assessments in a pair. 



Summary of Findings on Writing 

The teacher survey, teacher interviews at case study sites, and student interviews all 
support the finding that GALAXY students were writing more and were more aware of 
what they were writing. Rounding out the picture of GALAXY writing, FWL 
researchers also found evidence that GALAXY classrooms were not emplo)dng the full 
range of process writing, but rather they were concentrating on pre-writing activities. 
That is, students seemed to have many opportunities to talk about different sides of an 
issue and to work in small groups, but their GALAXY writing was more limited to 
quick-writes and jotting down faxes and ideas. The evidence suggests that developing 
arguments in writing was not an extensive part of the GALAXY experience for most 
students. 

The scoring rubric for both the writing prompts and the performance assessments 
reflected an increasing ability "to take a stand," that k, to marshal evidence in support 
of an argument; this is one of the key objectives of the GALAXY language arts 
curriculum. As one can see from looking at the rubrics in the appendices to this 
document, the rubrics were not designed to measure whether a student wrote more in 
later assessments, whether writing came more easily to the student, whether language 
mechanics such as spelling and punctuation improved, or whether sentences were 
somehow better constructed. 



^ 1676 students had both WPl and WP2. The sample size for PAl and PA4 was 640, and the sample size 
for PA2 and PA3 was 811. 
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Thus it is not surprising that the six writing tasks used in the evaluation produced 
mixed results in terms of writing gain: The writing prompts failed to show a statistically 
significant difference between GALAXY and comparison students, but the analysis of 
the performance assessments featured a statistically significant trend of increasing 
scores when the anomalous fourth task was excluded. One possible explanation for the 
mixed results is that, because they focused on students' ability to support a position 
with evidence, these six tasks may not have measured the principal area of writing gain 
during the fourteen-week GALAXY span. Other changes in writing behaviors and 
attitudes, such as willingness to write and ease of writing, might logically precede 
growth in ability "to take a stand." These behaviors and attitudes were not measured 
by the writing prompts and performance assessments. 

It may be possible to measure better significant changes in writing ability over the 
fourteen-week GALAXY period by focusing more on writing fluency, writing volume, 
and less complex elements of writing structure than "taking a stand." It is also likely 
that a longer theme cycle, which will allow teachers time for more process writing, will 
produce changes in students' writing that will be reflected in a number of different 
measures of writing ability. 
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VI. Teacher Attitudes and Practices 



Interventions to improve teaching will succeed only if classroom teachers embrace them 
as both useful and manageable. Our evaluation therefore carefully appraised the 
attitudes of teachers toward the implementation of the GALAXY curriculum and its 
several elements. 

There were three sources of data for this inquiry: a survey on attitudes and practices 
with regard to teaching language arts given before and at the conclusion of GALAXY; 
observations of teaching and repeated interviews with teachers at seven schools 
throughout the semester; and an extensive end of year survey on the use of GALAXY 
and perceptions about the usefulness of its components. 



Teacher Responses to GALAXY 

All of our evidence supports the conclusion that GALAXY was well-received by 
participadng teachers throughout the nation. Individual interviews, discussions at the 
in-service training institutes, and survey results all agree: virtually without exception, 
teachers expressed great enthusiasm for GALAXY language arts. 

One element of evidence derives from the end of year survey, when teachers were 
asked to assess the educational value of the GALAXY components, as having "great 
value," "some value," or "little value." The principal GALAXY elements were rated 
highly: the percentage of teachers judging that each element demonstrated " great value 
for educational purposes" were as follows: 95% for the broadcasts, 92% for the themes, 
87% for the classroom activities, and 94% for the GALAXY literature. Only 1 of 102 
respondents rated any of these components as having little value. 

Even fax machine use, which in several classrooms was marred by persistent technical 
problems, was rated as having "great educational value" by 77% of GALAXY teachers 
and some value by 19% others. Only four teachers rated it as of little value. 

Even more impressive were responses to an open-ended question asking, "How has 
GALAXY affected your teaching?" Of 88 respondents, we would rate 85 as clearly 
positive, one as neutral, and two who responded by saying they had some trouble 
incorporating the large number of possible activities into their teaching. 

The degree of enthusiasm is what is most remarkable, and it is exemplified by the 
following remarks: 

"GALAXY is exactly what I had hoped for to "perk up" my 20 years of 
teaching experiences. I was feeling very "burned out" with teaching — 

GALAXY has put me "back on track. " 

"I'm more motivated. Teaching is more fun. I can follow the excellent plans 
easily and enthusiastically and it saves me time. It's very thrilling — Keep it 
up!" 
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"It has made me afar better teacher!" 

"I am a 25 year veteran teacher. GALAXY has brought "new life" to my 
teaching. It is well planned and fun to do!" 

"GALAXY has been like a shot in the arm for me. I LOVE It!" 

"GALAXY helped me tremendously as an English teacher. GALAXY raised 
my self-esteem as it made me feel like more of a part of the technological world 
that we live in." 

"My students and I became very close as we discussed and explored the 
GALAXY themes. Thank you for making my school year so enjoyable!" 

"GALAXY has given me the burst of excitement I needed. After teaching 14 
years, I was experiencing "bum out" and this has excited me. Thank you!" 

"GALAXY has had a tremendous effect on my teaching. I have seen apathy 
among students in the last several years. Now I see active participation and 
success. The children's self image has improved. " 

"I have thoroughly enjoyed teaching this year because of GALAXY." 

"I love the GALAXY program!!" 

As these responses show, teachers were not simpiy dazzled by GALAXY'S array of new 
instructional inputs. Their enthusiasm arose from their ability to use GALAXY to teach 
in new ways and to engage their students with new depth. That enthusiasm was 
sustained by seeing their students' excitement and the learning that occurred as a result. 



Language Arts Teaching Attitudes and Practices. 

GALAXY was designed to assist teachers to bring the richness of a literature-based 
meaning-centered approach into classroom reality. While there is now widespread 
acceptance of the value of such an approach, teacher experience with this very different 
style of teaching is limited. Therefore, we anticipated differences among teachers in 
their initial attitudes and practices with regard to meaning-centered teaching. We also 
wanted to ascertain whether experience with GALAXY led to a change in those 
language teaching attitudes and practices. 

Teacher attitudes toward the teaching of language arts, together with self-reports of 
their typical practices, were assessed through survey instruments developed for this 
purpose. Questions were provided on both a pre-test and post-test basis. From these 
measures two scales were generated, a "Constructivist" scale and a "Language 
Fundamentals" scale. The two scales were retained as separate measures. In effect, 
some teachers believed in the utility of both meaning-centered, constructivist 
approaches and more traditional approaches, and they saw no inconsistency in 
combining them, while others saw them as opposite ends of a spectrum. 
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Initial attitudes and practices 

On average, teachers began with extremely pro-constructivist attitudes. Over 90% 
agreed or strongly agree with statements on the survey such as: 

• "Reading and writing activities, even for beginners, should focus on meaning and 
the functional use of written language rather than on exact word and spelling 
identification." 

• "When children encounter an unknown word while writing, they should be 
encouraged to spell it out as best they can, but continue writing." 

• "When children encounter an unknown word while reading, they should be 
encouraged to guess based on context and meaning, but continue reading." 

At the same time, 40% to 50% concurred with a number of statements focusing on the 
use of phonics amd more traditional methods for teaching language: 

• "It is best to teach reading and writing in the following order: 1) letters, sounds, 
spelling; 2) words, vocabulary, sentences; 3) paragraphs; 4) stories." 

• "Reading and writing skills are best developed through frequent drill and 
practice." 

• "When a child does not know a word when reading, he or she should be 
instructed to sound it out." 

A similar pattern displayed itself in those questions relating to actual practices. Very 
high percentages reported that they taught using the following meaning-based activities 
at least once a week: 



Table 15. Teacher Pre-test Survey Results Related to Meaning Based Activities 





iMMity , 


24(T(hiMip«r 

mmh 


Total 


Using integrated reading and writing 
techniques 


13% 


78% 


91% 


Use of cooperative learning 


19% 


57% 


76% 


Reading and responding to literature 


10% 


80% 


90% 


Reading aloud to students 


9% 


89% 


98% 


Students choose their own literature and 
read silently 


8% 


88% 


96% 



Percentages were notably lower on one key question, the use of thematic activities in 
teaching Language Arts, where only 63% reported using thematic instruction at least 
once a week. 
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However, there was also a high frequency reported for teaching language arts through 
more traditional or phonics-based methods. The following percentages applied to the 
use of these more traditional techniques at least once a week: 



Table 16. Teacher Pre-test Survey Results Related to Traditional Techniques 





W««kly 


2-5 Tlims p«r 
Week 


Total 


Teaching Language Arts using a basal 
reader 


17% 


47% 


64% 


Teaching students using phonics or word- 
attack strategies 


30% 


50% 


80% 



Attitudes and practices at the conclusion of GALAXY 

Pre/ post changes were modest on the constructivist scale items. Since teachers already 
had begun with nearly unanimous agreement about the virtue of meaning-centered 
language arts teaching, there was of course little room for positive change. 

However, on the key issue of thematic teaching, frequency did change markedly: those 
reporting teaching thematically from 2 to 5 days a week increased from 42% to 65%. 
GALAXY thus resulted in a change in a very key area of 23% of its teaching population. 
At the same time, it is notable that even with GALAXY'S carefully integrated thematic 
framework 21% of the teaching population reported teaching thematically less than 
once a week. 

Opinions about the use of phonics and traditional teaching techniques did not change in 
any notable fashion — ^but reported practice did. The use of basal readers declined from 
64% usage at least once a week to ^%. The reported use of phonics strategies declined 
from 80% at least once a week to 55%. Again, for a full one-quarter of GALAXY'S 
teaching population, a sigruficant change resulted. 

Even though there were major changes in the use of traditional and phonics-based 
approaches, there remained a clear bifurcation among the GALAXY teaching 
population, with about half the teachers continuing to use phonics and basal readers 
kirly frequently, and others using them infrequently (less than once a week). The use 
of phonics can be interpreted either as a marker for a somewhat more traditional kind 
of teacher or for a teacher who is eclectic in approach. We have no way to distinguish 
between these two groups from the attitude survey. 

Discussion 

In spite of wide variation among teachers in background, teaching style, ethnicity, and 
region of the country, no one could fail to be impressed with the virtually universal 
enthusiasm of teachers for the GALAXY program. We found that enthusiasm to be 
based on a belief that GALAXY had helped teachers introduce a more interesting and 
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effective meaning-based approach into the classroom. It also reflected their daily 
observations of the enthusiasm of children for the GALAXY experience and a sense that 
it was reaching all children, regardless of ethnic background, poverty, or gender. 

Will that enthusiasm be sustained over repeated experience with GALAXY? We cannot 
of course know. However, the portents are positive, in our judgment: the enthusiasm of 
children newly introduced each year to GALAXY should do much to sustain teachers' 
enthusiasm; the mastery of the fax technology, still daunting to some teachers, will 
grow and add renewed interest; and the wide variety of activity choices available to 
teachers through GALAXY can provide continuing interest and opportunity for creative 
teaching. 

There are other very important elements of teachers' success that arise from the use of 
galaxy's instructional technologies Because the television broadcasts are available 
on a precise schedule, a schedule around which the classroom activities are built, 
instruction is paced in participating classrooms; the likelihood of full utilization is thus 
increased. And in part because of the fax technology and the incorporation of 
classroom responses into both broadcast and fax communications, teachers feel 
themselves a part of a network, with all the social reinforcement that feeling provides. 

While we cannot quantify the impact of these factors, they add much to the probability 
of continued teacher use, and continued teacher enthusiasm, for the GALAXY 
experience. As their mastery of GALAXY teaching strategies increases with repeated 
cycles, there is reason to believe that student learning outcomes may be even further 
enhanced. 
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VII. The Role of Interactive Technologies in GALAXY 



Background 

GALAXY is the first large-scale educational innovation for elementary students to make 
use of the fax machine as an integral part of a language arts curriculum. An audio- 
conferencing system, the hoot 'n holler, was also used as an additional means of 
promoting interconnection among classrooms. Both of these interactive technologies 
make use of GALAXY'S satellite network. 

The dedicated fax system made it possible for students to respond to requests for their 
views on issues raised in the television broadcasts, to write to other classrooms on any 
topic, and to write to a student response bulletin in the expectation of publication. 
Some of these activities, thus, were structured opportunities for interacting provided by 
requests from the characters in the show, while others depended on the initiative and 
interest of students and teachers. 

The fax plays a prominent role in GALAXY because it has been assumed to be a major 
motivator for children's writing and a way to create a nationwide community of 
learners. Writing for fax transmission is writing about real topics to real people; this 
kind of writing can be an important element of meaning-centered instruction. 

The classroom fax capability also has other potentially motivating features: it is a new 
and manageable technology, and it breaks down the isolation of the single-teacher 
classroom, linking children to their counterparts in very different and distant places. 

The potential, then, seems very great. The use of classroom fax machines on a regular 
basis is a pioneering element of the GALAXY system. As a result, it was expected to be 
the element most likely to have initial difficulties, both in systems operation and in its 
integration into the instructional process. Our assessment has confirmed such initial 
difficulties, while also demonstrating the apparent instructional value of effective fax 
use. 



What Teachers Said About the Fax 

The contrast between interest in the fax's potential and the frustration of technical 
problems associated with it was reflected in teachers' responses to the open-ended end 
of year survey question asking for comments on the GALAXY technologies: 



Technical problems 

"One of the things that I was most excited about was the fax machine and its 
motivational force in writing, I was disappointed that it was operationally 
inconsistent." 
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"The fax machine became a source of frustration for both me and my 
students." 

"My students initially were excited about the fax machine. They became 
disenchanted as their many faxes continued to be unamwered." 

"Our fax was always messed up; we did fax within our school; we also were 
either taking or preparing for formal state testing. The class and I loved it, 
though." 

"Something has to be done to clear up the busy signals on ttie fax. This was 
very frustrating." 

"We tried the fax, but had trouble getting through so quit. Not being able to 
fax was a big disappointment. And I think we lost a lot of the educational 
benefit — the spontaneity of working a lot." 

"We worked daily and zveekly to get the fax machine to send faxes but we sent 
only one fax successfully. It has been most frustrating. " 

Classroom Interaction issues 

"The fact that our fax never seemed to work was very disappointing. The 
students attempted to fax materials to THE HOUSE and to our sister school to 
no avail" 

"Our partner school neve> faxed us; it really disappointed my kids. We did 
fax and they loved it and we received a few from other schools." 

"Response time is too long. Would it be possible to bypass the network and 
fax directly to partners? Students have forgotten or lost interest by the time 
they get a response." 

"Apparently our faxes didn't make THE Max. Students lost interest. We 
faxed quite often. I don't know what happened." 

"Fax had great possibilities — but didn't receive many answers." 

"Cooperative and enthusiastic fax partners would be nice. My kids were 
disappointed by the few responses we received." 

"Received only two of THE MAX—called and reported several times. Sent 
lots of faxes, rarely received from them. Kept writing to their pen pals, but 
were disappointed when they never received any responses from them." 

"Students really enjoyed writing faxes but became less enthusiastic, when 
faxes weren't returned to them in a more regular basis." 

"Our GALAXY partners did not communicate at all with us through the fax 
machine. My kids were very disappointed about this." 
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Educational value 

"The fax has also motivated the students to want to write more — they love to 
write to THE HOUSE and to others." 

"Using the GALAXY program has increased my students learning ability. 

They are eager to write their thoughts down. The idea of seeing their Ifaxed] 
work on television or in the newsletter makes them want to work harder." 

"The zvriting response rate is high when we receive faxes from other schools, 
but varies greatly from student to student." 

In personal interviews throughout the period of the study, the increasing frustration 
wiih the technical problems associated with the fax became clear. Teachers began with 
enthusiasm about the "real-world" character of the fax and its potential to link with 
teachers and students elsewhere. They also felt a lot of trepidation about their ability to 
use the technology, and they did not feel well trained in its use as the project started. 

Gradually, as delays and technical glitches continued to haunt the system, 
disappointment and frustration grew in many teachers. While they learned how to use 
the system, too often it was down. 

Another very widespread source of frustration was that classrooms frequently failed to 
get responses back from other classrooms they had faxed. Because of the way the 
system is configured, there is no automatic acknowledgment that a fax has been 
successfully sent, as there is in some commercial fax systems. This uncertainty and lack 
of reinforcement is a very serious problem for GALAXY, one that demands a technical 
solution and other steps. If system reliability becomes very high, and 95% or more of 
faxes go through, the problem would be largely solved. In addition, however, 
' classrooms should be urged to respond with an acknowledgment fax as soon as one is 
received, even before a substantive reply can be generated. 



Technical Operation and Reliability of the Fax 

The fax system became operational at different times in different schools during this 
first iteration of GALAXY. Some schools had installation problems that delayed their 
coming online for a number of weeks. In addition, the computerized control system 
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required to coordinate this unique dedicated satellite fax system required modifications 
during the early period of G ALAXY's operation. 

There also were classroom management challenges for some teachers associated with 
the newness of the technology. Problems resulted from the necessity for schools to 
adhere to a certain schedule for fax transmissions, a schedule required to accommodate 
the large number 'of faxes flowing over a satellite channel with limited bandwidth. 
There were difficulties in getting efficient repair of fax machines that were not working, 
delays exacerbated by the unfamiliarity of teachers with the machines and with ways to 
access the maintenance system (through an "800" number set up by GALAXY). The 
result of all these factors was markedly differential use of the fax. We asked each 
teacher to report on tlte number weeks the fax machine was operable during GALAXY'S 
14 weeks of activity. The results are in Table 17. 



Table 17. Teacher Report of Weeks of Fax Machine Operability 



} Number 


Pereent 


Always worked . j. 


8 


7.8 


inoperable 1-2 weeks 


26 


■25.5 


ftibperable S^weeka'" 


26 


25.5 


Inoperable ^WMks - 


12 


11.8 


lnop<Mab|e.74> ' 


27 


26.5 


Never 


3 


2.9 



Thus, among teachers responding to the end of year survey, in only 8% of classrooms 
did the fax system work from the beginning to the end of the project, while in another 
25% it worked except for one or two weeks. We must emphasize that these low figures 
include delays in start-up, not solely technical reliability after start-up. Nevertheless, 
the combination of the two meant a restricted ability in many classrooms to use this 
very interesting feature of GALAXY. 

We asked a related question at the end of the year: "How frequently did you use the 
fax?" Here we believe teachers were responding to its use after initial start-up, and so 
they were reflecting in most cases the degree to which its instructional uses had become 
an integral part of their teaching of the GALAXY curriculum. 
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Table 18. Teacher Report of Frequency of Fax Machine Use 





Number 


Percent 


Dally 


11 


10.7 


2*3 time per week 


48 


46.6 


Once per week 


20 


19.4 


Occasionally 


19 


18.4 


Not at all 


5 


4.9 



By the end of the period, as Table 18 shows, more than half the teachers were utilizing 
the fax two to five days a week, which was GALAXY'S intent. Twenty-three percent 
were using it only occasionally or not at all. (In some of these instances, the fax may not 
have been functioning.) 

In spite of these difficulties, 76% of teachers rated the educational value of the fax as 
"great," and only 4% of those who had used it rated the fax as having little value (see 
Table 19). 



Table 19. Teacher Report of Educational Value of Fax Machine Use 





Narober: 


' P«c«n#>v 


Groirwtjo^^ 


75 


75.5 




18 


17.6 




4 


3.9 




5 


4.9 



An even more enthusiastic endorsement was teachers' judgments of how much 
students liked the fax, as shown in Table 20. 

Table 20. Teacher Report of Student interest in Fax Machine Use 



1 


1 Number 


Percent;. 




j Lj 

86 


86.0 




6 


6.0 




0 


0.0 


Did not uie • - n -* 

1 


8 


8.0 
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P6rc6ived stud6nt ©nthusiasin goes down somewhat as we differentiate the various uses 
of the fax, for communicating within the school to other classrooms, with other schools, 
with the characters in the television programs, and with the student response bulletin, 
as illustrated by the data in Table 21. 



Table 21. teacher Rating of Student Enthusiasm About Faxing (Percent) 



Faxing Activity 


Veryv 

Enthusiastic 


Somewhat 

Enthusiastic 


Not 

Enthusiastic 


Did Not 
Work 


Did Not Do ! 


In-school faxing 


5Z5 


15.8 


4.0 


2.0 


25.7 


Faxing to other schools 


74.0 


17.0 


2.0 


0.0 


7.0 


Faxing to THE HOUSE 


81.2 


13.9 


0.0 


0.0 


5.0 


Faxing to THE MAX 


52.0 


34.3 


10.8 


0.0 


2.9 


Receiving in-school faxes 


46.0 


23.0 


27.0 


0.0 


27.0 


Receiving faxes from other 
schools 


69.0 


9.0 


3.0 


1.0 


18.0 


Seeing faxes on the B-Show 


80.6 


13.6 


2.9 


0.0 


2.9 



Note that the highest ratings went to faxing to the characters in THE HOUSE and to 
seeing their faxes on the broadcasts. The overall conclusion from the data in Table 21 is 
that faxing was a considerable hit in the great majority of classrooms. 



The Educational Results of Fax Use 

The fact that classrooms varied greatly in their use of the fax, in part because of the 
initial technical problems, provided an opportunity to relate use of the fax to learning 
outcomes. This section focuses on the gains in GALAXY classrooms in Vocabulary and 
Reading Comprehension. 

The overall correlation of the amount of classroom fax use and average student 
outcomes is .35 for Vocabulary, which is statistically significant (p=.002), and .10 for 
Reading Comprehension (which is not statistically significant). But this is not the 
whole story. Of particular interest was how different levels of classroom fax use relate 
to variations in student performance. 
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Table 22. Teacher Report of Frequency of Fax Machine Use and 
Corresponding Vocabulary Scale Score Gain 





Percent 


Average Vocabulary 
Scale Score Gain 


Dally 


10.7 


14.6 


241 tbnas pantvMk 


46.6 


16.1 


Oncaparwaak 


19.4 


11.5 


Occasionally -v'i 


18.4 


9.3 


MoC at all ■ ■■ . ■ 


4.9 


6.7 



There are striking differences between those using the fax at the suggested rate of two 
or more times a week and those below that rate. As Table 22 shows, in classrooms 
where the fax was used only occasionally, the average gain score on Vocabulary was 
only 9.3 points whereas those who used the fax two or more times per weeks gained 14 
or more points. Frequency of fax machine use can be seen as an index of commitment 
to GALAXY, and, therefore, it appears that those classrooms with the higher interest 
had the greater student gain. These data would*suggest that the use of the fax may be a 
very important part of GALAXY'S success. Where it was rarely used, students 
performed only slightly better than those in non-GALAXY classrooms at GALAXY 
schools (9.3 vs. 8.7). Where it was used as intended, students did very well. Indeed, in 
classrooms that frequently used the fax, students gained somewhat in NCE and 
percentile scores when compared with the national norms. 

We cannot rule out the possibility that some teachers who used the fax little were in 
other respects not very much involved in using the GALAXY curriculum or that they 
may have been ineffective for other reasons. Nevertheless, the differences found on this 
particular factor suggest a plausible interpretation that the use of the fax is integral to 
the educational success of GALAXY students. 

Use of the Audio-Conferencing System 

galaxy's "hoot 'n holler" system links all classrooms through a "party line" 
telephone that is operable throughout the day. Teachers can use it to contact other 
classrooms by simply getting on the line and announcing throughout the system a 
desire to speak to a particular classroom. If that classroom is listening, then they can 
respond. GALAXY can also use it for more structured audio-conferences for training 
and administrative purposes. 

What teachers said about the hoot ‘n holler 

The purpose of the hoot 'n holler was never clearly articulated by GALAXY, and this is 
reflected in comments from the teachers: 

"The hoot 'n holler often served as a distraction." 
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'7 loved the hoot ‘n holler!" 



"Students were frustrated with the hoot ‘n holler, because it was difficult to 
connect with classrooms (time zones, classes not having them turned on, etc.) 
and the clarity of the voices. It was used less as the program continued. " 

As Table 23 indicates, the system was used infrequently by the great majority of 
teachers. 



Table 23. Teacher Report of Frequency of Hoot ‘n Holler Use 





Number 


Percent 


Dally 


3 


3.0 


2-3 time oer week 


9 


9.1 


Once per week 


5 


5.1 


Occasionally 


46 


46.5 


Notrtair^';* 


36 


36.4 



Surprisingly, 66% of teachers attribute some educational value to the system, although 
only 20% say it has great value. These figures are the lowest for any component of the 
GALAXY program, by far. Teachers do say, however, that students like the system, as 
shown by the data from the end of year survey in Table 24. 



Table 24. Teacher Report of Student Interest in Hoot ‘n Holler Use 





Ntmbet 


Percent 




24 


25.8 




17 


18.3 




2 


2.2 




50 


53.8 



Discussion 

Most audio-conferencing systems used for education or training are centered around 
scheduled activities, such as training sessions or discussions of topics where sharing 
information is essential. Audio-conferencing systems are also used to provide audio 
feedback from many sites as part of teleconferenced training, where the television 
program is a one-way broadcast. All systems that operate seriously in these ways have 
online operations managers who serve to schedule linkages and to moderate 
discussions. 
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galaxy's hoot 'n holler, however, was largely unscheduled and unintegrated into the 
educational process. It was used as a part of one teleconference centered on technical 
training, and otherwise it was left to the spontaneous use of teachers and/or students. 
It was ill-suited to the latter purpose. 

Interviews with teachers have made it abundantly clear that they do not like having 
their classes interrupted each time someone in the network is trying to use the system. 
The result was that many turned the system off, so they were unable to hear requests 
that might have been directed toward them. The audio quality of the system was poor, 
probably because of low-cost equipment at the classroom level. And there were no 
effective means to arrange a conference with another classroom, aside from a 
commercial telephone call or trading a series of faxes, both too cumbersome for most 
teachers. 

It is clear that the hoot 'n holler would not be missed if eliminated and that it is not 
workable in its present arrangement. It could, however, be utilized as an important part 
of scheduled teleconferences, both for in-service training and for linking classes 
throughout the system. Its potential for this kind of use has not been tapped and is 
worth exploring. 

Summary of GALAXY’S Interactive Role 

GALAXY'S unique interactive capability, in particular a system-wide fax network, is 
one of its most interesting and exciting features. The use of the fax appears to be a 
strong motivator of student interest and encouragement for writing, and it serves to link 
students to a wider world. 

It also presents technical challenges that must be solved. For many classrooms in the 
demonstration phase, the fax came to work reliably enough to permit its use several 
times a week or even daily. Those classes produced the largest gains in reading scores, 
as measured by standardized tests. While follow-up throughout a period when the fax 
system is working more reliably is called for, the continued utilization of the fax as an 
integral part of GALAXY seems fully warranted. 
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VIII. Program Components 

GALAXY language arts had a number of program components which all contributed to 
the total experience for the GALAXY students. These components engaged the 
attention of students and helped teachers to implement the GALAXY curriculum. They 
include the broadcasts ^bout the children at THE HOUSE; the teacher resource materials 
including the suggested classroom activities; the use of small group instruction; the 
GALAXY literature books (each teacher had six books per theme); the take-home 
magazine. The SECOND STORY; the response bulletin, THE hAAX, that printed faxes 
received from student; and the staff development efforts of GALAXY, particularly the 
two Institutes for teachers. Each will be discussed in turn in this chapter. 



Broadcasts 

The broadcasts are the linchpin of the GALAXY program, and they are highly 
successful. Both the teachers and the students give the broadcasts almost universal 
approbation. As Table 25 shows, 99 out of 104 teachers (95%) who returned the end of 
year teacher survey thought the broadcasts had great educational value. An even 
higher percentage of teachers (97%, 101 of 104) reported that their students liked the 
broadcasts a lot. It looks like a win-win situation for GALAXY: high educational value 
and great kid appeal. 

Teacher comments 

Teachers at the case study sites consistently awarded the broadcasts high marks, 
praised their success at introducing and reinforcing the language arts themes, and 
recognized how the characters at THE HOUSE provided a common frame of reference for 
discussion of the themes. Repeated viewings of the broadcasts were a common 
occurrence, especially early in the cycle. 

Several commented on how they appreciated the ethnic diversity of the cast and how 
the characters appealed to all types of students (rural, urban, Spanish-speakers, and all 
ethnic groups). Students identified strongly with the characters, believing that "the kids 
are just like us." 



Table 25. Teacher Rating of the GALAXY Broadcasts 



Educational Value 





Number 


Percent 


Great Value 


99 


95.2 


Some 


5 


4.8 


Littki Value. 


0 


0.0 


Old Not Use 


0 


0.0 
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How Much Students Liked 



Number 


Percent 


Uikbd a Lot >i . 


101 


98.1 


Uk^ Somewhat^ 


2 


1.9 




0 


0.0 




0 


0.0 
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Even reshowing the broadcasts rated well with the teachers, as Table 26 shows; only one 
thought that using the VCR to reshow a broadcast had little educational value. 



Table 26. Teacher Rating of Reshowing the Broadcasts 



Educational Value 





Number 


Percent 


1 Liked '8 Lot- 


89 


86.4 


Uked ;;; 


11 


10.7 


DW Not Uko - 


1 


1.0 


Dkf Mot Use r .,'i‘ 




1.9 



Kow Much Students Liked 





Nimiber 


Percent 


Great Value 


90 


88.2 


Some Value 


9 


8.8 


Little Value 


1 


1.0 


Did Not Use 


2 


2.0 



Comments from teachers about the broadcasts in the end of year teacher survey all 
substantiate their high regard: 

“The program turned the students and me on! The children forgot that it 
was 'only a story' and became personally involved with the characters at THE 
House. In addition to the above, I believe it improved the children's 
relationships with each other as they became more sensitive to the feelings of 
others." 

“Students love the TV program! They seemed to lose motivation to respond to 
fax invitations. " 

"I related a lot of our work to the themes and broadcasts... Jannelle was a 
great model, as well Katy & Miguel." 

“GALAXY has brought new ideas and activities to my classroom this year. 

I've been just as excited as the students to view the broadcasts, discuss the 
problems, and predict outcomes." 

“I can say that students look forward to the broadcasts, the characters are real 
to them, they get involved with the problems, and transfer what they have 
learned in other subjects to GALAXY." 

“The broadcasts and themes were well received, and looked forward to, by all 
my students. The enthusiasm for activities varied." 

Student comments 

Students at the case study sites were clearly enraptured by the broadcasts, as seen in 
Table 27. Part of the interview process involved playing "red light-yellow light-green 
light." Students were given three different colored circles on a stick to hold up as an 
indication of how they felt about different GALAXY components. Nearly all the 
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students interviewed (98.8%) awarded the broadcasts their green light signal ("Way to 
Go, GALAXY"). They were somewhat less enthusiastic about writing about the 
problems at THE HOUSE. While many students awarded GALAXY a green light on this 
aspect of the broadcasts, several students selected the red light ("Stop GALAXY") and 
the yellow light ("It's okay GALAXY") primarily because of frustration due to technical 
fax problems and secondarily because of difficulty with writing. Nevertheless, most 
students were excited to see their work and the work of their friends on THE HOUSE and 
perceived their faxes as being written for a real audience (i.e., the various characters in 
The House). 



Table 27. Student Ratings of The House 





Qr««n 












% 


• 










The Shows from THE HOUSE 
(N=81) 


98.8 


80 


0.0 


0 


1.2 


1 


Writing about the problems 
at THE HOUSE (N=79) 


50.6 


40 


36.7 


29 


12.7 


10 



Students were asked to rate the various parts of the broadcast. As seen in Table 28, the 
two most highly regarded features of the broadcasts were Mira's stories (88.0% green 
lights) and Carl's Cool Carl Comics (90.7% green lights). The feature that earned the 
highest "disapproval rating" was Mary Alice's stories (26.6% red lights). 



Table 28. Student Ratings of Parts of the Show 





9nm 

• 




House rap 


66.7 


50 


20.0 


15 


13.3 


10 


Simon's rap 


53.3 


40 


36.0 


27 


10.7 


8 


The Dramas at THE HOUSE 


53.3 


40 


36.0 


27 


10.7 


8 


Mary Alice's Stories 


30.7 


23 


42.7 


32 


26.6 


20 


Mira's stories 


88.0 


66 


8.0 


6 


4.0 


3 


Carl's "Cool Carl Comics" 


90.7 


68 


9.3 


7 


0.0 


0 


Faxes being read on the show 


77.3 


58 


16.0 


12 


6.7 


5 



Students at the case study sites were so taken with the broadcasts that they cited what 
they had learned from them when FWL researchers asked about the most important 
thing they had learned during GALAXY: 
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"I learned from THE HOUSE that sometimes I was similar to Carl and I 
haven't faced all my problems yet like Carl — ^so I learned a lot about myself" 

“What I learned when Joey and Carl read Jannelle's diary and invaded her 
privacy was how you need to forgive your friends. Listen to yourself, and 
don’t be hard on others because they did something bad to you. Give friends 
another chance." 

“All the time you have a problem, you should tell them and ask for advice. I 
don't like to tell about my problems, and tell people to "bug off." But then I 
don't have help. If you tell people, then you get lots of suggestions. All the 
characters at THE HOUSE had problems and we helped solve some of them." 

"I learned from the shows that other kids have problems - once everyone forgot 
my birthday — and you can learn about things you don't know and can relate 
to the kids. I learned to respect others." 

Improving broadcast schedule and instructional use 

While the broadcasts were highly successful, some technical and instructional areas for 
improvement have been identified. One technical problem was that some teachers were 
confused by the broadcast schedule. In addition, most teachers found the two day 
broadcasting cycle for each show to be too short, given the complexity of school's 
calendars. 

FWL researchers observed at the case study sites that broadcasts were not always used 
as intended or to their best advantage. In some classrooms the broadcasts were 
overemphasized to the exclusion of other elements of GALAXY. In addition, often there 
was little or no debriefing of the broadcast prior to beginning other GALAXY activities. 
It was also noted that the Spanish broadcasts tended to be used for review purposes 
with the Spanish-speaking students rather than for the initial showing. 

Some teachers made suggestions on their end of year surveys for small changes to the 
broadcasts: 

"I'd like the characters on THE HOUSE to speak a little slower (Miguel and 
Jannelle)." 

"...we are focusing on writing correctly. It would help if during the 
broadcasts, the students used great correct grammar, punctuation, spelling, 
and capitalization." 

"Put the words to the raps on the screen for all broadcasts." 

"I would have liked to have seen the TV programs in advance so that I could 
coordinate the activities better for my group." 

These suggested changes are all relatively minor and reflect the fact that teachers are 
really thinking about how they can use the broadcasts m dieir instruction. 
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Summary 



Like the themes which they illustrate, the broadcasts resonate well with teachers and 
students, and they are a powerful way of grabbing students' attention. FWL researchers 
encountered no negative comments at all about the content of the broadcasts. 

Some changes of a technical nature could be made by GALAXY to improve teachers' 
use of the broadcasts. (1) Simplify the broadcast schedule and timing to reduce teacher 
confusion and increase teacher ease in showing a live broadcast. (2) Revise the schedule 
so that each broadcast is shown for at least three days. (3) If possible, program an 
automatic taping of the broadcasts at each site. 

Several small changes in the activity guide would enhance instructional use of the 
broadcasts: (1) Provide teachers in the activity guide with a set of two or three 
suggestions for follow-up questions for each broadcast. (2) Provide teachers of Spanish- 
speaking students with assistance in using the Spanish version of the broadcast more 
effectively in class (e.g., modeling through videotapes and suggesting ideas in the 
activity guide). 



Student Activities and Teacher Resource Materials 

Teachers at the case study schools assigned great value to the GALAXY activity guide, 
giving it high marks for its ease of use and engaging and motivating activities. 
Classroom visits revealed that the activities helped teachers and students meet many of 
the important GALAXY goals. For example, the p^tivities appeared to meet the needs of 
students of varying ability levels. Teachers selected activities that were most 
appropriate for their students and selected activities that incorporated a variety of 
instructional modalities (e.g., art, drama and role play, listening, writing, reading). 

Teachers, on the whole, demonstrated an ease in adapting the activities to their own 
particular needs, teaching styles, and interests. For example, while some teachers 
emphasized activities that focused on writing performance, others used those activities 
that provided students with opportunities to engage in activities that encouraged the 
use of critical analysis skills (e.g., problem solving, comparing, contrasting, predicting). 
Many of the activities achieved their intended outcome in providing students with the 
opportunity to build on personal experience and prior knowledge, to engage in 
dialogue with their peers, and to express a point of view. 
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Table 29. Teacher Rating of the GALAXY Activities 

Educational Value How Much Students Liked 





Number 


Percent 


Uked a Lot 


79 


76.7 


Liked Somewhat 


24 


23.3 


Did Not Uko 


0 


0.0 



Usefulness 





Number 


Percent 


Great Value 


89 


87.3 


Some Value 


12 


11.8 


Little Value 

1 


1 


1.0 



Number 


Percent 


Very Useful 


96 


94.1 


Somewnat Useful 


5 


4.9 


Not Very Useful 

1 ' 


1 


1.0 



The end of year teacher survey confirmed that the GALAXY activities were really 
popular with the vast majority of the teachers. The data in Table 29 shows that 87% 
rated them as having great educational value, and 77% thought their students liked the 
activities a lot. The GALAXY lesson plans /activities that appeared among the teacher 
resource materials were considered essential by the teachers at the case study schools, 
and the teachers surveyed concurred in this assessment: Ninety-six of the 102 teachers 
(94%) rated the sug^ted activities as very useful. 



Table 30. Percent of Approximately 100 Teachers 
Rating the GALAXY Teacher Resource Materials 



Resource , ' 


Rating of the Usefulness in Helping You to Implement 
OALAXY With Your Students 


















Show synopses 


84.0 


15.0 


1.0 


1.0 


List of critical issues 


82.2 


17.8 


0.0 


1.0 


Literacy strategies and demonstrations 


86.1 


13.9 


0.0 


0.0 


Literacy references 


76.5 


21.4 


2.0 


0.9 


Written copy of the raps 


65.7 


31.3 


3.0 


2.0 


Theme bibliographies 


75.2 


23.8 


1.0 


0.0 


Teacher Resource Book 


87.5 


11.5 


1.0 


0.9 


Literacy Activities Handbook 


83.5 


16.5 


0.0 


0.9 
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As Table 30 above indicates, all of the elements of the teacher materials rated very 
highly with thi^ GALAXY teachers, especially the Teacher Resource Book and the 
literacy strategies and demonstrations. 

Teacher open-ended comments 

On the end of year teacher survey, many of the GALAXY teachers commented on how 
important the suggested activities and the Teacher Resource Materials had been to 
them: 



"The bibliography was fantastic. I loved the teaching activities which 
promoted higher-level thinking skills." 

"The Literacy Guide has really helped me in areas that I was hesitant or 
unsure about. The biggest problem for me has been the hectic pace trying to 
fit in all the fun and wonderful activities in a two week time span." 

"I have been given creative, usable activities that help reach my wide range of 
learners. " 

"The many activities your materials have provided and literature made 
available have enhanced my delivery — I lo-ve your activities!" 

"I tried many new ideas that I have read about in the Literacy Activity 
Handbook and the theme units. What great ideas! I think GALAXY made 
my teaching more interesting and meaningful." 

"I wanted to implement a literature-based reading program into my classroom 
but I felt really tied to the basal. Because of GALAXY we used the basal for 
only GALAXY activities (because it included stories from the bibliography). I 
also have enjoyed the writings activities with my students and 1 know my 
students have enjoyed them much more than classes from previous years." 

"GALAXY gave me motivation and enthusiasm as I looked forward to doing 
the GALAXY activities and using the equipment." 

"Instead of inventing new and cute activities, these are already planned out. I 
just have to prepare and teach them. This frees my time up considerably." 

Summary 

It is hard to imagine a teacher's guide and list of suggested student activities that would 
be more successful than GALAXY'S. There are several other goals of GALAXY 
instruction that would benefit from further emphasis in the teacher resource materials: 
They include a discussion of providing students with sufficient choice about what and 
how they learn and with opportunities to demonstrate their unique strengths and 
attributes as related to cultural heritage. Additionally, since very few of the activities 
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explicitly emphasize the use of the various GALAXY technologies (e.g., the fax, the hoot 
'n holler, and the computer), new activities making use of these technologies for 
communicating beyond the classroom should be incorporated into the teacher's guide. 



Small group Instruction 

Many of the GALAXY activities encourage the use of small groups for instructional 
purposes. As Table 31 below shows, all but one of the teachers who answered the end 
of year survey thought that GALAXY helped students to work productively in groups. 
Similarly, doing "a better job of working with their peers to solve problems" was rated 
third highest in importance by the teachers from a list of thirteen behaviors. More than 
80% of the teachers, as Table 31 shows, thought that most or some of their students 
reasoned about issues better with their peers. Students' working together had a positive 
effect on their interaction with their classmates. 



Table 31. Percent of Teachers Rating GALAXY Small Group Instruction Effects on 

Student Behavior 



How well dOM GALAXY h«ip studonts to: Very Woli 


Somowhat 

WeU. 


Not Vary 
. ..Wall 


NotatAtt 


Work productively in collaborative groups 


57.4 


41.6 


1.0 


0.0 



Since GALAXY bi^n, my etudents: 


Troefbrr^ 


TrueRSic:. 


^Inie focai. 




Do a better job of working with their peers to 
solve problems 


35.9 


47.6 


14.6 


1.9 


Reason about issues with their peers 


38.8 


41.7 


18.4 


1.0 



Several of the GALAXY teachers at the case study sites demonstrated excellent skill at 
using small groups. However, repeated observations of GALAXY classrooms indicate 
that those teachers who struggled with small group instruction from the outset of the 
program continued to experience difficulty throughout. In fact, some of these teachers 
found ways to adapt the activities so that they could be used for whole class instruction. 
The primary problems with small group instruction involved student confusion about 
role and task. This confusion would inevitably require the teacher to spend more time 
on classroom management issues than on instruction. Students in these classes 
recognized the problem and were able to voice their frustration when asked their 
opinion. On the other hand, in those classrooms where small group instruction was 
successful, students were able to articulate their appreciation about learning with and 
from their peers. 

On the end of year survey, teachers commented on the success of the cooperative 
groups and on the collateral effects they had observed among their students: 
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"The debates and other cooperative activities were superb for strengthening 
analytical, social, and psychological development in students." 

"They work cooperatively very well and enjoy the diverse cultures in the 
room." 

"The students enjoyed coming to my classroom for the cooperative GALAXY 
groups. " 

"...the cooperative learning groups worked well and I plan to give my 
students many opportunities to share ideas orally in groups and for the whole 
class next year to develop some of the necessary skills to become good readers 
and writers. " 

"GALAXY helped the students work cooperatively which gave me time to 
focus on those students who were having special problems. Language Arts 
took on a whole new meaning and look. Awesomel!" 

"GALAXY has helped or affected my teaching by enabling me to be more 
tolerant of noise when the students are working together to solve problems. 
Additionally, my students are working more to help each other and to solve 
problems in all subject areas." 

"I am more inclined to use cooperative learning and to encourage children to 
interact with each other. I have thoroughly enjoyed this challenge and hope 
that the Language Arts focus will be extended." 

"The cooperative groups worked excellently. Activities weren't even looked 
upon as reading/writing activities. I found myself looking forward to 
GALAXY classroom days." 

Some students who were inteirviewed at the case study sites singled out the group 
experiences as "the most important thing they had learned from GALAXY." 

"Learning is fun and it's not boring. I learned how to work in groups and 
cooperate and take turns, that's all. And I learned how to write stories." 

"I learned how to work in groups. I used to not like working in groups. They 
kind of slow me down. Now I learned to get some of their ideas. " 

Summary 

GALAXY small group instruction was successful for most classrooms, but some 
teachers, particularly those who had not used cooperative groups before, experienced 
some difficulty. Two recommendations regarding small group instruction are 
proposed. The first is to provide teachers with additional staff development support in 
facilitating small group experiences. The other is to provide teachers with additional 
alternative activities that could be taught in either small groups or large groups. 
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GALAXY Literature 



"I have learned reading the books has worked. My students have enjoyed the 
stories and we have had many wonderful discussions from the stories." 

— comment from GALAXY teacher 

High quality children's literature was an important component of the GALAXY 
program. The literature was selected to relate to the seven GALAXY themes, while 
providing a choice for differing levels of difficulty and teacher preference. During the 
GALAXY demonstration phase, six books of the teacher's choice were provided for each 
theme — forty-two for the entire cycle. 

The objective for including the literature was to develop literacy and enhance self- 
esteem by helping students understand and debate opposing viewpoints, express their 
personal strengths, and analyze how the books might relate to their own lives. Briefly, 
GALAXY wanted to bring high quality children's literature into the classroom, to 
integrate the GALAXY themes and children's Uterature, and to engage students in 
reading, thinking about, and discussing the themes that underlie the curriculum. 

Teacher reflections 

The literature component of GALAXY received high marks from both teachers and 
students at the case study schools. Teachers greatly appreciated the way the books 
complemented the GALAXY themes and their outstanding quality. They are being 
used as intended to engage students in reading about and discussing the themes that 
underlie the curriculum. Demonstration phase teachers have expressed an interest in 
having more GALAXY books than currently allowed, especially those that are read by 
Mira on the broadcast. Several felt that a single book could be used only as a "read 
aloud." Others found creative ways to use one book for instruction (e.g., small group 
read-alouds with students reading to each other). Bilingual teachers have noted that 
they prefer literature that is not a translation from English. Apparently, the translation 
process elevates the reading level of the text. 



Table 32. Teacher Rating of the GALAXY Uterature Books 



Educational Value 



1 . 


Percent 


Great 


94 


91.3 


Some Value 


8 


7.8 


Uttlo Value ;. 


1 


1.0 


Did Not Use vr.::li 


0 


0.0 



How Much Student* Uked 





Nundier 


Percent 


Uked a Lot*' 


94 


90.4 


likdki: Somswnai^ 


10 


9.6 




0 


0.0 


DW.Not..Use.Hi.‘:>*:S 


0 


0.0 
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Table 32 confirms the impression from the case study schools: over 90Percent of the 
teachers thought that the literature had great educational value and that their students 
liked the books a lot. 

Teacher open-ended comments 

On the end of year teacher survey, teachers mentioned the GALAXY literature and how 
much they and their students enjoyed the reading: 

"The literature selections are great." 

"As a teacher, the literature component was impressive. I used more 
literature in my lessons on a regular basis." 

"The literature books are by far the best selection of multicultural stories that I 
have ever read to my classroom." 

Student comments 

Students at the case study sites were asked several questions about 'he literature 
component. First, they were asked to rate two aspects of the program; how they liked 
reading the books and stories and hov^ they liked their teacher reading GALAXY books 
aloud. As seen in Table 33 the majority of students (83.8%) greatly enjoyed the books. 
Those few who gave this component a yellow light did so primarily because they 
thought some of the books were "too easy." Students who were less enthused about 
having the books read aloud to them by their teachers were critical of the style and 
expression their teacher used. 



Table 33. Student Ratings of GALAXY Literature 





Qn 

% 


MNt 1 

* 




Reading the GALAXY 
books and stories (n=80) 


83.8 


67 


16.2 


13 


0.0 


0 


Yoiir teacher reading 
GALAXY books to you 
(n=49) 


72.0 


59 


19.5 


16 


8.5 


7 



Secondly, they were asked to select their favorite GALAXY literature and to explain 
why they picked that book. They selected books easily, and with great excitement, and 
were often challenged by selecting their one favorite book. They loved having the 
opportunity to talk about these books. Their comments show that they were able to 
make connections between the GALAXY themes and the literature. In addition, 
students offered comments that show they appreciated how books can help them deal 
with everyday problems as well as be sensitive to their personal needs and interests. 
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They offered the following reasons for a favorite GALAXY book that demonstrate the 
kind of connections students were able to make between the themes and the literature: 

Faithful Elephants — "It made me think about things, like how unfair it is for 
animals and how lucky we are to be humans" 

Rag Coat — "The book was about people needing people. When she wore the 
coat to school, everyone teased her but she liked the coat because the material 
came from everyone." 

One Foot, Now the Other — "First the grandpa taught the grandson how to 
walk and then when the grandpa forgot how, the grandson taught him. People 
need people and nothing else. We'd be lonely without people. " 

Big Orange Splot — "Because Mr. Plumbean was different from the others, 
he was true to himself and did not let others bug him. He was brave and he 
was a leader. He encouraged people to follow their dreams. " 

Big A1 — "At first he seemed ugly and mean — but they just couldn't see the 
other side of him — his inside was nice." 

"I liked how he saved all the little fishes. He was a hero!" 

Other reasons for favorite books had to do with the books' way of addressing an issue 
or topic of special importance to them. These books and reasons include: 

Brave Irene — "I respect bravery. She wasn't a he-man, but she was really 
brave." 

Angel Child, Dragon Child — "I like to learn about history and things about 
different races. I like to read books about prejudice because I'm into peace and 
I like to find new ways to help make peace. " 

The Pain and the Great One — "I have a brother. He's little and I'm big. 

He gets treated differently. It helps you think of things you can do with a 
brother." 

The Gold Cadillac — "It's a warning book — about where not to go when you 
are Black." 

La Calles es libre — "/ liked how the pictures looked like my home in Puerto 
Rico. It reminds me of my cousins. I was happy there." 

The Leaves of October — "It makes me think about people who are homeless 
and I'd like to help them and they are nice people, not crazy people." 

Summary 

Both teachers and students were entranced by the GALAXY literature books, and they 
all saw clear connections to the GALAXY themes. As GALAXY moves beyond the 
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demonstration phase, some planning might be done to facilitate schools' access to and 
ordering of the GALAXY literature books. In addition, the teacher's guide might 
include additional ideas on how to use a single book in a classroom. 



THE SECOND STORY 

Although The Second Story was received with mixed reactions during the first part of 
the language arts cycle, by the time the last copy of the magazine was distributed both 
teachers and students seemed be more comfortable with the publication and had found 
ways to use it. Many of the activities appeared to have been tried both in school and at 
home. Several students expressed their appreciation for having their own "piece of 
GALAXY" to take home. Student ratings of THE SECOND STORY were largely favorable. 
Most (69.1%) gave the magazine a green light, 24.7% gave a yellow light and 6.2% gave 
a red light. 

Teacher comments 

Teachers at the case study sites reported that they believed the magazine had 
encouraged their students to read, although they also reported that the magazine 
format and "busy-ness" contributed to making reading more difficult. In addition, 
there was considerable uncertainty among teachers regarding how to use the magazine 
as a tool for in-class instruction, for involving parents, and for homework assignments. 
These comments were generally confirmed by data from the end of year survey, as 
shown in Table 34. 



Table 34. Teacher Rating of The Second Story 



Educational Value 





Number 


Percent : 


: Great Value ■ 


60 


58.8 


Some Valuo 


39 


38.2 


little Valua^f . 


2 


2.0 


Did Not 


1 


1.0 



How Much Students Liked 





Number 


Percent 


Liked a Lot ^ 


67 


66.3 


Liked Somewhat 


34 


33.7 


Old Not like 


0 


0.0 


Old Not Use 


0 


0.0 



One teacher's comments suggest some of the problems with the student magazine: "THE 
Second Story: the kids read them, some discussion, but we couldn't fax, and no one 
looked up the book references. We would not read them all together because of two 
languages." 
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Summary 



The Second Story has the potential for further encouraging reading among GALAXY 
students. But data from teachers in the demonstration phase suggest that GALAXY 
should consider making The Second Story an optional purchase component of GALAXY. 
In addition, GALAXY should reconsider the purpose and use of the magazine and 
should perhaps move to a less "busy" format. 



THE IVIAX 

Ratings of the student publication, THE MAX, were largely favorable among students at 
the case study schools. Most (65.3%) gave The Max a green light, with yellow (24.5%) 
and red (10.2%) far behind. Generally they were pleased to see their work and the 
work of their friends in this publication. However, both students and teachers were 
often disappointed when their work was not included in THE MAX despite many 
submissions. In addition, teachers seemed uncertain about how to use the one copy 
they received in their fax machine with the entire class. Generally it was posted on a 
bulletin board or put in a special folder. Teachers had developed few strategies for 
using The Max for instruction. 



Table 35. Teacher Rating of The Max 



Educational Value 





Number 


Percent 


Great Value 


41 


41.8 


Some Value ' “ ^ 


43 


43.9 


Little Value . , ; 


7 


7.2 


Did Not Use— 7 


7.2 



How Much Students Liked 





Number 


Percent 


Liked a Lot ' 


48 


49.5 


Liked ^hew^tat 


40 


41.2 


DWNotUKO 


2 


2.1 


DiffNot^Use-. 7 


7.2 



The end of year survey shows that teachers were not nearly as excited about THE MAX 
as they were about other components of GALAXY. As Table 33 above shows, over 80% 
of the teachers thought that it had great or some value. And an even higher margin 
thought that their students liked it. However, The Max did not score as well as other 
program components such as the broadcasts, which 95% of the teachers thought had 
great educational value. When asked separately about faxing to THE MAX, teachers 
were more positive about their students' attitudes, as indicated in Table 36. 
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Table 36. Teacher Rating of Student Enthusiasm About The Max (Percent) 



Activity 


Very 

Enthusiastic 


Somewhat 
Enthusiastic _ 


Not 

Enthusiastic 


Did Not Do 


Faxing to THE hAAX 


64.0 


26.0 


3.0 


7.0 


Receiving THE MAX . 


52.0 


34.3 


10.8 


2.9 



Teacher Comments 

Teacher comments on THE MAX centered on the difficulty some of them had in knowing 
whether their students' faxes were getting through and the disappointment that 
students suffered when their names and/or work did not appear in THE MAX: 

"Apparently our faxes didn't make The Max. Students lost interest. We 
faxed quite often. I don't know what happened." 

"Received only two o/THEMAX — called and reported several times." 

"Since we were not sending faxes successfully, the issues of THE MAX lost 
some of their appeal. However, when we had our first author published in 
issue 6 A the excitement returned." 

"Need a way to recognize all faxes sent to THE HOUSE or THE MAX. 

Students lost some interest when they did not see their work printed or used." 

"The Max needs to include more student writings, at least one from each 
school for every issue." 

Summary 

Student interest is piqued when they see their own work or the work of their friends 
and acquaintances in THE MAX. However, this is counterbalanced, to some extent, by 
the number of classrooms that tried to fax to THE MAX, only to be disappointed when 
their work did not appear. They were left in a quandary about whether ^eir faxes had 
been received, and some students were "tumed-off" to faxing by the number of 
unsuccessful attempts. Given this uneven experience with THE MAX, GALAXY might 
consider the elimination of this component. If it is retained, there should be some 
mechanism to guarantee better representation of all students, larger print, and a less 
cluttered appearance. 



Staff Development 

The Institutes won extremely high praise from teachers, as Table 37 shows. They 
appreciated 'he quality and value of the workshops and presentations. They felt that 
the Institutes contributed greatly to their ability to teach GALAXY. However, teachers 
at the case study sites expressed the belief that they would have benefited from 
additional ongoing support and reinforcement throughout the cycle for what they had 
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learned at the Institutes. In addition, most teachers agreed that they would have 
appreciated having more hands-on experience with the various hardware components 
and having technology sessions taught by other teachers. They also noted that it wa^ 
especially difficult in classrooms of at-risk students for teachers to be away from the 
classroom in order to attend the Institutes. They provided vivid descriptions of the 
management problems they faced during and after their absence. 



Table 37. Teacher Rating of Usefulness of Language Arts institutes (Percent) 



Usefulness of the following in helping you to I ^ ,, - , 

Implement GALAXY with your students j 


Somewhat 

Useful 


Not Very 
Useful 


Did Not Use 


Los Angeles Institute 


84.2 


8.4 


2-1 


5.3 


Atlanta Lnstitute 


81.3 


11.5 


0.0 


7.3 



On the end of year teacher survey, teachers expressed their strong feelings that the 
teacher Institutes had been valuable for them: 



"I feel that the Atlanta staff development was the best I've every attended." 

"Especially helpful to meet and re-meet teachers and staff in the GALAXY 
Program. Sharing thoughts and ideas with other educators was a strong part 
for me!" 

"Training was superb, especially concerning literacy strategies. The scope of 
specific, as well as general open-ended approaches, was developed and 
presented. " 

"The LA Institute was wonderful but Atlanta was more meaningful because 
we already had implemented the program and more direct questions and the 
sharing with the teachers was very practical as well as useful. But — both 
institutes provided very valuabl. ~taff development." 

"The Institutes and resource materials have aided me in teaching a more 
holistic approach in the language arts area." 

Summary 

The GALAXY Language Arts Institutes are so successful that they should continue as a 
regular part of GALA)^, perhaps on a regional basis. GALAXY might want to tape 
outstanding Institute presentations for broadcast during the year. Additionally, the 
Institutes should provide time for GALAXY teams (e.g., principals, teachers, resource 
teachers) to engage in joint planning. 
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IX. Conclusion 



The GALAXY Classroom Project was developed as a nationwide reform effort to infuse 
new curricula into schools, to spark the interest of teachers and students in learning, 
and to make a significant difference in the educational lives of students who 
traditionally have been labeled "at-risk." 

The first phase of the GALAXY demonstration. Language Arts for Grades 3-5, achieved 
a number of unequivocal successes. It brought to a cross-section of America's 
economically disadvantaged students a classroom experience that connects reading and 
writing to their own concerns, knowledge, and experiences, and that respects their 
substantial intellectual capabilities. 

GALAXY reached its diverse multicultural learners in a way that generated 
extraordinary student enthusiasm. That enthusiasm was shared by teachers, who in 
numerous cases reported that the experience had brought new life to their professional 
work. GALAXY has had an impact beyond language arts; it has changed the way many 
teachers view the capabilities of their students. GALAXY'S unique interactive 
component, the use of the fax machine, fueled that enthusiasm, despite technical 
problems. 

The changes that GALAXY generated had a positive impact on student learning; 
GALAXY students achieved statistically significant greater gains on both Vocabulary 
and Reading Comprehension measures when compared with the gains of students in 
similar classrooms. Writing measures for GALAXY students abo showed a statistically 
significant pattern of gain. As experience develops further in subsequent iterations of 
GALAXY, further positive impact should be seen. 

The evaluation team can point to a number of factors that makes GALAXY work: 

• The integrated thematic curriculum links broadcasts, literature, instructional activities, and 
the use of the fax. The themes have personal relevance to children's lives and provide a 
context for reading and writing, a common frame of reference for teachers and students. 

• Access to the technology in their own classrooms — TVs for viewing GALAXY broadcasts, 
VCRs for reviewing shows, fax machines for communicating with other students and with 
the producers — provides students with opportunities to write for real audiences and to 
become part of a nationwide community of learners. Further, they gain positive 
reinforcement for using technology - a good experience for many of the students and their 
teachers. 

• A key component is the staff development for the teachers in how to use the curriculum, its 
literacy strategies, and its activities and how to integrate technology into the curriculum 

Perhaps the impact of GALAXY car\ best be summed up in the words of a GALAXY 
student from Orangeburg, SC: "You can be smart, but it [GALAXY] can help you get 
smarter. " 




Conclusion 
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Appendix A. Technical Appendix 



David J. Pasta 
DMA Corporation 



NCE Scores in Generai 

The Normal Curve Equivalent, or NCE, is a kind of standard score that is designed to be 
similar to percentile scores. As a standard score, it is approximately Normally 
distributed. It is set so that, for the reference group (in tikis case the sample used for 
establishing norms), it has a mean of 50 and a standard deviation of about 21.06. The 
standard deviation is actually set to that an NCE of 1 corresponds to a percentile of 1 
and NCE of 99 corresponds so a percentile of 99. By convention, the NCE scale is 
truncated at 1 and 99, even though as a standard score it has no theoretical upper or 
lower limits. 

As a standard score, it is reasonable to manipulate NCE scores by addition, subtraction, 
and calculating an average. It is not appropriate to manipulate percentiles in this way. 
That is because a change from a percentile of 5 to 10 represents a much larger change 
than a change from 40 to 45, say. Near the middle of the distribution, a small change in 
test score allows you to move past many others. In contrast, near the ends of the 
distribution, a large change in test score is needed to move past the same number of 
individuals. 



NCE Scores on CAT/5 Tests 

The NCE scores available from CTB for the CAT/5 tests administered are based on a 
national norming sample from a few years ago. Even assuming the national normmg 
sample was representative of the target national population, the population has 
undoubtedly changed since the norming sample was tested. To the extent the nature of 
those changes are Imown, they should be considered in interpreting the NCEs. 

The students tested have an average NCE around 40, which corresponds to a percentile 
of around 32. This means that, as expected, the students tested tend to score lower than 
the national norming sample. 

The national norming sample is cross-sectional. That means that, for example, the third 
graders and fourth graders were tested the same year. There is no assurance that the 
third graders would, as a group, have the same distribution of scores the following year 
(when they are in fourth grade) as the fourth grader*: who were actually tested. In fact, 
if test scores are steadily increasing or steadily dec /easing over time, the cross-sectional 
norms can be misleading when considered longitidinally. 

Consider the following simplified example. Suppose third graders in the national 
norming sample get an average score of 350 and fourth graders an average score of 400. 
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Suppose further that test scores in general are declining 10 points per year. Then the 
next year the third graders in the national norming sample, who would now be in the 
fourtii grade, would be expected to average 390 (not 400). Furthermore, even if it is 
known that the students will average 390 that does not say a great deal about the 
distribution of gain scores. It is unlikely in the extreme that every student will increase 
by 40 points. No doubt many students will increase by substantially less or even 
decline (due largely to measurement error), and others will increase by substantially 
more. The only way to know the distribution of gain scores would be to have a 
longitudinal norming study. Unfortunately, test publishers rarely have the time and 
resources to perform a longitudinal norming study. 

Instead of longitudinal ncrming studies, users of norm-referenced tests are usually 
forced to rely on assumptions. Usually it is assumed that the norming sample, al^ough 
cross-sectional, is representative of the distribution to be expected among the tested 
students. Furthermore, it is assumed that the growth among students is symmetric 
about its mean, so that half the students grow more than the midpoint of the norms and 
half grow less. Unfortunately, even with these assiunptions, it is still not possible to 
calculate the average growth for a group of tested students unless their score 
distribution is centered at the same place as the national norming sample. 

For students who score lower on average than the national norming sample, empirical 
studies have shown that they tend to grow less them indicated by the national norms. 
Unfortunately, we don't know how much less. Thus the expected average gain for the 
students in this study lies between a zero gain in scale score (a drop in NCE or 
percentile) and a zero gain in NCE or percentile (an increase in scale score). 



Scale Scores 

The scale scores reported for the CAT/5 are designed to be an equal-interval scale 
across the grades and abilities of elementary and secondary education. That is, a 10 
point increase from 300 to 310 is designed to be comparable to a 10 point increase from 
800 to 810. Furthermore, the scale scores are standard scores, based on the Normal 
distribution, and therefore can be manipulated by addition, subtraction, and calculating 
averages. 



Floor and Celling Effects 

If a test is much too difficult or much too easy for a student, the resulting score is 
questionable. For each test, there is a lowest obtainable scale score (LOSS) and a highest 
obtainable scale score (HOSS). Students who score only about as well as would be 
expected by chance would be assigned scale scores at or near the LOSS, the "floor" for 
scale scores for the test and level. Given a more appropriate level of the test (an easier 
one), those students might get about the same scores or might get substantially lower 
scores. They are unlikely to get substantially higher scores. Similarly, students who get 
all the items correct would be assigned the "ceiling" scale score, the HOSS. If these 
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high-scoring students were given a more appropriate (harder) test, • e could estimate 
their true scale score much better. 



Standard Error of Measurement 

Associated with any test is measurement error. This represents the fact that in 
hypothetical repeated administrations of the same test to the same individual, the 
resulting score will not be exactly equal every time but tend to fluctuate around the 
individual's "true score." Tlie "true score" is nothing more or less than the (theoretical) 
mean score that would be obtained in repeated administrations. In practice, one caimot 
expect to estimate the score by repeated administration because the individual would 
remember items if the administrations were close in time and because the individual 
might have learned things if the administrations were not very close in time. 

There are various ways to estimate the measurement error, however, and CTB provides 
an estimate of the Standard Error of Measurement (SEM) associated with the scale score 
for each subtest. The SEM varies substantially according to the actual test score. 
Students scoring near the middle of the test range (getting around half the items correct) 
will tend to have the smallest estimated SEM. Students assigned the lowest or highest 
obtainable scale scores will have the largest estimated SEM — a value often many times 
larger than the smallest SEMs. For the CAT/5 tests, the estimated SEMs are quite near 
their minimum value for a wide range of test scores but inevitably increase dramatically 
as the ceiling or, especially, the floor of the test is approached. 

There are more scale scores associated with large SEM at the low end of the scale 
because of guessing. On a 25-item multiple-choice test with five choices for each item, a 
student attempting all items can be expected to get about 5 correct purely by chance. 
However, it is quite possible that students guessing randomly on each item might get 6 
or 7 or even 8 or more items correct purely by chance. On the other hand, they might 
get only 2 or 3 correct by chance and other items that were correct might reflect true 
knowledge. The calculation of the SEM takes into account the likelihood of obtaining 
the resulting score purely by chance and so is noticeably large for scale scores associated 
with answers below or near the guessing threshold. In contrast, only the top few scores 
have highly inflated SEMs. The highest score (HOSS) has a large standard error of 
measurement because it is hard to estimate how much higher the scale score would 
have been with a harder test. Similarly, an individual getting only one or two items 
incorrect might actually know the corre^'t answer (and merely have mismarked the test) 
or might be able to do many similar questions but have misinterpreted the item or 
otherwise had unusual difficulty with it. In other words, the very large SEMs at the 
high end of the sco*" - distribution tend to be associated witli "careless errors" or no 
errors at all. 
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Gain Scores 



We performed our analyses with gain scores, the difference between the post-test score 
and the pre-test score. In the case of two test scores, as we have here, the gain score 
analysis is equivalent to a repeated-measures Analysis of Variance and is also 
equivalent to an analysis of individual growth curves. Both of these alternative 
analyses are appropriate when three or more time points are available. There was a 
time when some authors argued against analyzing gain scores, but their arguments 
have been convincingly answered in the more recent literature. In particular, the 
analysis of post-test score using the pre-test score as a covariate has been shown to be 
fraught with problems. 

Although the reliability of the gain score is low relative to the reliability of the pre-test 
and of the post-test, that is more due to the fact th?d the actual differences in growth 
among individuals (true variance) tend to be small when the pre-test and post-test are 
only a few months apart. Note that the reliability, defined as the true variance divided 
by the sum of the true variance and the measurement error variance, approaches zero as 
the time between the pre-test and the post-test approaches zero because the true 
variance also approaches zero. For current purposes, suffice it to say that the gain score 
is an unbiased estimate of the growth between the two tests and that it is as reliable as 
possible. 

The '‘Reading Total” Score 

The "Reading Total" scale score is simply the average of the Vocabulary scale score and 
the Reading Comprehension scale score. The standard error of measurement (SEM) for 
the Reading Total score is provided by CTB. It appears to be calculated as one would 
expect under the usual assumption of uncorrelated measurement error across subtests, 
namely as half the square root of the sum of squared SEMs. Thus if the SEM for 
Vocabulary were 10 and the SEM for Reading Comprehension were 8, their minimum 
values, the SEM for the Reading Total would be at its minimum value of 7 (half of the 
square root of 100+64, or about 13/2). 



The SEM for the Gain Score 

The SEM for the gain score is not reported by CTB, but can be estimated under the 
simplifying assumption of uncorrelated measurement errors between the pre-test and 
post-test. Under that assumption, the SEM for the gain score can be estimated as the 
square root of the sum of squared SEMs for the pre-test and the post-test. The 
minimum estimated values are given in the following table: 
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Table A.l Minimum Estimated SEM of Gain Scores 



Test 


Minimum SEM for 
Pre- or Post-test 


hfiriimiim SEM for 
Gain Score 


Vocabulary 


10 


14.14 


Reading 

Comprehension 


8 


11.31 


Reading Total 


7 


9.90 



This means that the gain score for an individual is at best known within a standard 
error of 10 or more. 



Weighting Scores According to Estimated Variance 

Scores with high standard error of measurement (SEM) are not estimated very precisely. 
This ancillary information needs to be considered in the analysis. One relatively crude 
approach would be to eliminate from the analysis any individuals with "unacceptably 
large" SEMs. This approach creates several difficulties. One is that the cutoff for 
elimination is necessarily arbitrary. Another is that individuals with acceptable SEM for 
one subtest but not the other either must be eliminated from both analyses (thereby 
eliminating perfectly good information) or the sample used for each subtest (and the 
Total Reading score) must be allowed to be different. 

A better approach is to downweight the scores with high SEM. It can be shown that 
using weights inversely proportional to the variance of the score (the square of the 
SEM) results in optimal estimates (in the sense of Best Linear Unbiased Estimates). For 
convenience, the weights were defined relative to uie minimum estimated variance. 
Thus individuals with estimated gain score SEM equal to the minimum for the test were 
assigned a weight of 1.0. Individuals with very high gain score SEM were assigned a 
very low weight. 

Student-Level Analyses of Matched Classrooms 

Many of the analyses comparing treatment to control were done for two distinct sets of 
students. The first set of students represent the most tightly matched groups of 
students and the second set represents the vast majority of students. The most tightly 
matched groups of students were the comparison classrooms in GALAXY schools and 
the corresponding GALAXY classrooms (in the same grade). There were 29 comparison 
classrooms in 22 school-grade combinations across 21 schools (one school had 
comparison classrooms in two grades). Each of these 22 school-grade combinations was 
matched with at least one GALAXY classroom at the same grade, for a total of 26 
GALAXY classrooms. There were, in general, other GALAXY classrooms in the schools 
but they were at different grades. Thus a total of 55 classrooms in 21 schools were 
matched in 22 school-grade combinations. 
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The analysis of these matched classrooms was done at the student level. Logically, the 
difference between treatment and control was estimated for each of the 22 school-grade 
combinations and then the 22 estimates combined. If all of the school-grade 
combinations had included exactly the same number of GALAXY and comparison 
students with valid test scores (a completely balanced design), the difference between 
GALAXY and comparison would be tibe difference between the means for the two 
groups of students. Because the number of students varied somewhat from school to 
school, the actual estimate deviates somewhat from the simple mean. The estimated 
mean scores for the treatment and control students in unbalanced designs are 
sometimes called "least-squares means" or "estimated population means." We will 
usually term them estimated means to distinguish them from the simple sample means. 

For the matched-classrooms analysis, it is appropriate to perform the analysis at the 
student level because each school-grade combination included in the analysis had both 
treatment and control students. That is, the treatment was crossed with the 
experimental unit (school-grade combination). This provides a relatively powerful 
analysis because the treatment effect is estimated after controlling for school and grade. 
On die other hand, many students had to be omitted from the analysis, including all the 
students from non-GALAXY comparison schools and all the students from GALAXY 
schools where there were no comparison classrooms. 



Classroom<Level Analyses of Most Students 

Besides the student-level analyses of students in the matched classrooms in the 21 
schools, additional analyses were performed on the vast majority of classrooms. These 
analyses were performed at the classroom level. In other words, the difference between 
GALAXY and comparison was evaluated by comparing the gains among comparison 
classrooms as a group with the gains among GALAXY classrooms as a group. Where 
the distribution of gains among GALAXY classrooms was shifted compared to the 
distribution of gains among comparison classrooms by more than could be expected by 
chance, GALAXY gains were declared to be significantly greater than comparison gains. 
Note that these analyses include the matched classrooms in the 21 schools. 

The analysis as described does not control for possible grade, school, or district (region 
of the country) effects. In fact the analyses checked for grade effects (and grade-by- 
treatment interactions) and found none. There were significant school and district 
effects, but the analyses did not control for them. The decision not to control for school 
and district effects was based on a series of analyses that considered the effect of 
controlling for school and district in the classroom-level analyses. Although there is 
substantial variation from school to school and district to district (region to region), the 
variability is only about twice as large as the class to class variability within schools. 
Because controlling for school would necessitate eliminating many classrooms, 
including all classrooms in non-GALAXY schools, statistical power to detect treatment 
differences was improved by not controlling for school effects. Similarly, even 
controlling only for district (region) would have meant eliminating those districts that 
had no comparison classrooms. The small increase in error variance (class to class 
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variability) from ignoring school and district effects was more than offset by the 
increase in classrooms includible in the analysis and the simpler form of the analysis. 



Students Included in The Analysis 

We omitted students \Vho had invalid CAT/5 tests, who took any test in Spanish (the 
SABE 2 or Writing Prompt or Performance Assessment), or who took the wrong level 
CAT/5 test. We also eliminated students who did not have both CAT/5 subtests. 

Item Response Theory Scoring 

The tests were scored using Item Response Theory (IRT) rather than simple number- 
correct raw scores. This scoring method takes into accoimt the difficulty of an item 
when evaluating the effect on the score of a correct or incorrect answer. The effect is 
that the standard error of measurement is smaller with IRT scoring than with traditional 
number-correct scoring. That is, the resulting scale score is generally more accurate 
when using IRT scoring than when using number-correct scoring. One effect is that two 
individuals with the same number of correct items may have different scale scores. 



Correlation Between Pre-test And Gain 

The correlation between the pre-test score (initial status) and gain score (difference 
between post-test and pre-test) is of considerable theoretical interest. A positive 
correlation indicates that students who start higher tend to gain more ("the rich get 
richer"), while a negative correlation indicates that students who start lower tend to 
gain more ("the poor are catching up"). If the correlation is approximately zero, this 
means that the gains are not systematically related to initial status ("a rising tide lifts all 
boats"). 

Unfortimately, the sample correlation between the pretest and the gain score is not an 
unbiased estimate of the corresponding population correlation because of measurement 
error and the associated "regression to the mean" effect. The bias is such that the sample 
correlation tend*' to be negative when the true correlation is zero. The folit^wing 
simplified example illustrates this. Suppose that the true gain is the same for every 
individual and so the gain is unrelated to the initial status. Individuals who (by 
measurement error) have a relatively low pretest score will tend to have a higher post- 
test score (and greater gain). Similarly, individuals with a relatively high pre-test score 
because of measurement error will tend to have a lower post-test score. Both of these 
effects lead to a negative correlation between pre-test and gain score. 

Given an estimate of the reliability of the test scores, however, it is possible to obtain an 
unbiased estimate of the population correlation between the pre-test and the gain score. 
From the CAT/5 Technical Bulletin 1 0une 1992, p. 49), we get the following values for 
reliability as measured by Kuder-Richardson Formula 20 (KR20): 
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Table A.2 Reliability by Test and Grade for Pre-test/Post-test 





3. 


Grade 

4 




Vocabulary 


.78 


.78 


.69 


Reading Comprehension 


.80 


.79 


.80 


Reading Total 


.87 


.87 


.84 



Using VI and R1 to denote the variance and reliability of the pretest and V2 and R2 the 
corresponding values for the posttest, and using Cll' to denote the covariance between 
the pretest and posttest, the estimated reliability of the gain score, R12, is calculated as: 

V1*R1 + V2*R2-2*C12 

R12= 

VI + V2 • 2*C12 

Choosing reliability values as indicated for the pre-test and post-test, we obtain the 
following reliabilities for the gain scores: 



Table A.3 Reliability of Gain Score 



Reliability of Pre- 
and Post Test 


Renabllltyof 
Gain Score 


Vocabulary 


.78 


.046 


Reading Comprehension 


.80 


.126 


Reading Total 


.87 


.192 



These low values are typical for gain scores over relatively short intervals of time (in 
this case, 17 weeks). TTiey reflect the fact that the true variability between individuals in 
gain score over a short period of time is small relative to measurement error. As the 
time interval between tests is reduced to zero, the reliability of the gain score is also 
necessarily reduced to zero. 

Having an estimate of the reliability of the gain scores and of the pretest score permits 
the calculation of a reliability-adjusted estimated correlation between the pre-test and 
gain: 



Table A.4 Estimated Correlations Between Pre-test and Gain 





Sample Correlation 


Estimated Rellabllty- 
Adjusted Correlation 


Vocabulary 


-.3315 


-.0453 


Reading Comprehension 


-.2555 


.0975 


Reading Total 


-.2156 


.0216 
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The reliability-adjusted correlations between pre-test and gain are all very small and 
vary in sign. This implies that there is no measurable systematic relationship between 
pre-test score and gain score: test score improvements are approximately the same for 
all levels of initial status. 



Reliability of Writing Prompts and Performance Assessments 

The Writing Prompts (WP) and Performance Assessments (PA) were scored using a 
similar process. A scoring rubric was developed and "anchor" papers provided during 
training sessions for the readers. The WP scores ranged from 1 to 6 (with some 
indicated as unscorable) and the PA scores from 0 to 5. During the actual readings, 
papers were read in batches of the same test (WPl, WP2, or PAl through PA4). A 
random sample of 10% of the tests were then scored by a second reader. If the two 
readers differed by more than oi e point, they were then scored by the chief reader. 

For the writing prompts, most of the scores were between 2 and 4, with only a handful 
of the scores at 6. The distribution of scores across the two writing prompts was as 
follows: 



Table A.5 Distribution of Writing Prompt Scores 









Unscorable 


123 


21 


Score 1 


362 


6.3 


Score 2 


1207 


21.0 


Scores 


2591 


45.2 


Score 4 


1126 


19.6 


Scores 


291 


5.1 


Score 6 


36 


0.6 


Total 


5735 


100.0 



The distribution of scores across the four performance assessments had a similar shape: 
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Table A.6 DIstIbution of Performance Assessment Scores 





# 


% 


Score 0 


36 


0.8 


Score 1 


413 


8.6 


Score 2 


1450 


30.3 


Scores 


1977 


41.3 


Score 4 


743 


15.5 


Scores 


166 


3.5 


Total 


4785 


100.0 



The score assigned for analysis purpose was the median of the available scores. For the 
papers that were assigned two scores that differed by 1, the median is the average of the 
two scores, giving rise to scores that end in ".5" in the analysis. For the 51 papers where 
the two readers differed by more than one point, the median turned out to be the value 
assigned by the third reader (the chief reader) in all but one case. That is because the 
chief reader almost always agreed with one of the readers or assigned a score in 
between the two. In the one exceptional case, the first reader gave a score of 4, the 
second reader a score of 2, and the chief reader a score of 5. The median score, 4, was 
used for analysis. 

An analysis of variance was performed to check for various possible influences on test 
scores. The readers were at one of two tables during the scoring, but no significant 
difference between the two tables was found. Also, no significant order effect was 
foimd (whether readers were tougher or easier on papers when they were being scored 
first or were part of the 10% rescoring sample). There were, however, significant 
differences among the 16 readers. The following estimated "reader effects" relative to 
the chief reader were obtained from the papers that were scored twice: 
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Table A. 7 Estimated Reader Effects for Writing Prompt and Performance Assessment 

Scores 



Reader 


Effect 


t 


P 


120 


-.629 


- 3.64 


<.001 


122 


-577 


-357 


<.001 


114 


-.319 


-203 


.043 


127 


-298 


- 1.80 


.073 


113 


-228 


- 1.49 


.140 


121 


-219 


- 1.40 


.160 


125 


-.121 


- 0.78 


.440 


112 


-.113 


- 0.69 


.490 


116 


-. 0 % 


- 0.68 


.500 


115 


-.089 


-058 


560 


123 


-.072 


- 0.44 


.660 


124 


-.057 


-057 


.710 


111 


.023 


0.06 


.950 


126 


.023 


0.15 


.880 


no 


.246 


1.74 


.082 



It can be seen that most readers tend to give slightly lower scores than the chief reader. 
The two lowest-scoring readers assign significantly lower scores. The other readers 
give scores that are reasonably close to each other and to the chief reader. A synopsis of 
the reader effects might be as follows: 



120, 122 



Low about .5 points 



114, 127 

111,112,113, ^ 

115,116,121, 
123,124,125,126 y 

110 



On the low side of normal range (.25 points?) 



In the normal range 



On the high side of normal range (.25 points?) 



Although it would be possible to analyze the WP and PA after making adjustments 
based on this ANOVA, no such analyses were performed. The reader effects are fairly 
small and are unlikely to result in any significant biases in the analyses performed. 



From the ANOVA and related analyses, it is possible to estimate the interjudge 
reliability of the WP and PA tests to be in the range .59 to .79. Tliese are reasonable 



O 



ppendices 



31 



89 



interjudge reliabilities but unfortunately they serve only as an upper bound on the 
test-retest reliability that we need to use to calculate a reliability-adjusted estimate of the 
correlation between pre-test and gain. 

Perhaps more instructive is to estimate the test-retest reliability that would produce a 
zero estimated correlation between pre-test and gain. That calculation yields reliability 
figures of .202 (for the gain from PA2 to PAS), .275 (for the gain from PAl to PA4), and 
.303 (for the gain from WPl to WP2). In other words, if the test-retest reliability is 
aroimd .2 to .3 for these tests, then the estimated correlation between the pre-test and 
the gain is zero. 

If the test-retest reliability of these tests is greater than the .2 to .3 range and approaches 
the interjudge reliability of .6 to .8, then the estimated correlation between pre-test and 
gain is increasingly negative. 

In light of the relatively low correlation between pre-test and post-test (.22 for PA23, .33 
for PAl 4, and .37 for WPl 2). it seems reasonable to conjecture that the test-retest 
reliability is fairly low for these tests. Moreover, a true negative correlation between 
pre-test and gain could be due in part to floor and ceiling effects in the restricted range 
of scores available. Accordingly, it seems unreasonable to reach any conclusions based 
on the possibly negative correlation between pre-test and gain for the PA and WP tests. 
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Appendix B: Psycho-Social Instrument 



Key; R=Reading, W=Writing, S=About School, SC=Harter's Scholastic Competence* 







What i Am Like 






For Teacher to Fill Out: 












School: 




Teacher. 




Class: 




For Student to Fill Out: 
Student Name: 






(first) 




(last) 






Age: 




Birthdate: 






Boy Girl 








month 


day 


year 


(circle one) 




Really 


Sort of 




Sort of Really 


True 


True 








True True 




for me 


forme 




for me for me 






Sample Sentences 




t 


(a) Q 


□ 


Some kids would rather play 
outdoors in their spare time 


BUT 


Other kids would rather watch TV. | 


□ □ 


1 


(b> Q 


ID 


1 Some kids do very well at all 
kinds of sports 


BUT 


Other kids don’t feel that they are [ 
very good when it comes to sports. I 


□ □ 


i 


i.[4l 


\m 


I Some kids like to read in their 
free time 


BUT 


Other kids would rather not read in I 
their free time. 


nil] 


1 " 


^ a 


la 


1 Some kids wish they could do 
more writing in school 


BUT 


Other kids wish they could spend 
less time writing in school. 


na 


1 w 


3. [£] 


la 


1 Some kids feel that they are 
very good at their schootwork 


BUT 


Other kids wony about whether I 

they can do the schoolwork 
assigned to them. ^ 


as 


1 sc 


-a 


la 


1 Some kids do not like the 
1 things they do in school 


BUT 


Other kids like the things they do I 
in school. 


n a 


1 ® 


3.0 


la 


1 Some kids feel that what they 
1 read In school Is boring 


BUT 


Other kids enjoy what they read in I 
school. 


[3]0 


1 « 


6.0 


la 


1 Some kids hate to change 
1 something they have written 


BUT 


Other kids like to change what they I 
write. 


n a 


1 w 




la 


1 Some kids feel that they are 
1 just as smart as other kids 
1 their age 


BUT 


Other kids aren’t so sure and 
worider If they are as smart. 


[2]0 


1 sc 


a 0 


la 


1 Some kids like the way they 
1 are taught to read 


BUT 


Other kids do not like the way they 
are taught to read. 


n E 


1 " 


9. 0 


la 


1 Some kids feel they are good 
1 readers 


BUT 


Other kids feel they don’t read very 
well. 




1 " 



‘Thanks to Professor Susan Harter of the University of Denver for facilitating our use of elements from the 
Self-Percqjtion Profile for Children, whid (he developed. 
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10. 


a a 


Some kids feel they are good 
at writing about their ideas 


BUT 


Other kids wish they were better at 
writing about their ideas. 




w 


11. 


a a 


Some kids are pretty slow in 
finishing their schoolwork 


BUT 


Other kids can do their schoolwork 
quickly. 


aa 


sc 


12. 


aa 


Some kids are not so sure that 
they like going to school 


BUT 


Other kids like going to school. 


aa 


s 


13. 


aa 


Some kids don’t like to talk 
about a good book they have 
read 


BUT 


Other kids like to tell their friends 
about a good book they enjoyed. 


aa 


R 


14. 


aa 


Some kids hate to write 


BUT 


Other kids feel that writing is fun. 


aa 


W 


15. 


a a 


1 Some kids often forget what 
they learn 


BUT 


Other kids remember things easily, j 


a a 


sc 


16. 


aa 


1 Some kids feel they learn a lot 
in school 


BUT 


Other kids feel they don’t learn 
very much in school. 


aa 


s 


17. 


a a 


1 Some kids feel that it is 
1 important to read well 


BUT 


Other kids feel that it is not 
important to read well. 


aa 


R 


18. 


a a 


1 Some kids are proud of the 
1 things they write in school 


BUT 


Other kids wish they could feel 
more proud about what they write. 


aa 


w 


19. 


aa 


1 Some kids do very well at their 
1 classwork 


BUT 


Other kids don't do well at their 
classwork. 


aa 


sc 


20. 


aa 


1 Some kids that it is not 

1 important to write well 


BUT 


Other kids feel it is important to 
write well. 


aa 


w 


21. 


aa 


1 Some kids have trouble 
1 figuring out the answers In 
1 school 


BUT 


Other kids can almost always 
figure out the answers. 


aa 


sc 


22. 


a a 


1 Some kids would like to do 
1 less reading in school 


BUT 


Other kids wish they could spend 
more time reading In school. 


aa 


R 


23. 


a a 


1 Some kids do not like the way 
1 they are taught to write 


BUT 


Other kids like the way they are 
taught to write. 


a a 


W 
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Appendix C: Writing Prompt 1 



Directions To Teachers On Administration Of The Writing Prompt 

Important!! Please Refrain From Looking At The Description Of The W riting 
ACTIVITY Until The Time You Administerthe W riting Prompt, thanks! 

Note to Teachers: Students will be scored on their ability to write their ideas clearly 
and persuasively. They will not be scored on spelling or form. 

Please proceed as follows; 

• See that each student receives one copy of the "writing packet" and one piece 
of yellow blank paper. The yellow paper is a planning page for organizing the.r 
ideas and making notes. 

• Ask students to fill in the information on the cover page — name, age, 
birthdate (day, month, and year). 

• Read aloud the section entitled "Instructions" or ask students to read the 
instructions aloud . 

• Answer any student questions about the directions. 

• Ask students to open their booklet to the second page, where they will find the 
writing task entitled "Writing a Letter." 

• Read aloud the description of the writing task, and ask students to follow you 
by reading silently. 

• Take a few minutes to answer their questions about the task. Try to be sure 
they understand the task. 

• Remind them about using the yellow paper to help them get started. 

• When you feel they xmderstand what they are supposed to do, please say, "You 
may now begin. You have 25 minutes to complete the task" and start timing 
the 25 minute period from that moment. 

At the end of 25 minutes, please collect the packets. You may discard the yellow 
sheets of paper. Write your school, teacher, name, and class in the space provided on 
the cover sheet of each student's packet. If you teach only one Galaxy c-ass, put in 
"A." For each additional Galaxy class that you teach, use "B," "C," "D," etc. 

Hand these packets to your Data Coordinator. 

Thank you very much! 
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Writing Activity 



For Teacher to Fill Out: 

School: Teacher: Class: 

For Student to Fill Out: 

Student Name: 

(first) (last) 

Age: Birthdate: Boy Girl 

month day year (circle one) 



Instructions 

You will have 25 minutes to write a letter. Read the 
assignment carefully and think about it before you 
begin. Be sure to respond to every part of the 
assignment. Remember tliat you can use the 
planning page to make notes and organize your 
ideas. 

Write your letter on the lined pages. Make your 
response as thoughtful and complete as possible. 
When you are writing your letter, be sure that your 
handwriting is clear. 

If you finish before time is called, you should go 
over your work again and change anything that you 
think will make your writing better. 
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Text of First Writing Prompt 



Writing a Letter 



A spaceship from another planet has landed on 
Earth. The creatures in the spaceship seem to be 
friendly and have let the space center in the United 
States know that their ship must return to their 
planet. Some scientists want to keep the spaceship 
on Earth and study the creatures. Others think the 
creatures should be allowed to return to their own 
planet. 

Write a letter to the director of the space center. Tell 
what you think should be done with the creatures 
and their spaceship. Be sure to give reasons to 
support your ideas. 



^Based on the fourth grade persuasive writing prompt of the National Assessment of Educational Progress (NAEP). 
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Appendix D: Writing Prciwot 2 



Directions To Teachers On Administration Of The Writing Prompt 

IMPORTANT!! PLEASE REFRAIN FROM LOOKING AT THE DESCRIPTION OF THE W RTTING 
Activity u ntil The Time You AdministerThe W rtting Prompt. Thanks! 

Note to Teachers: Students will be scored on their ability to write their ideas clearly 
and persuasively. They will not be scored on spelling or form. 

Please proceed as follows: 

• See that each shident receives one copy of the "writing packet" and one piece 
of yellow blank paper. The yellow paper is a plarming page for organizing their 
ideas and making notes. 

• Ask students to fill in the information on the cover page — name, age, 
birthdate (day, montl't, and year). 

• Read aloud the section entitled "Instructions" or ask students to read the 
instructions aloud . 

• Answer any student questions about the directions. 

• Ask students to open their booklet to the second page, where they will find the 
writing task entitled "Writing a Letter." 

• Read aloud the description of the writing task, and ask students to follow you 
by reading silently. 

• Take a few minutes to answer their questions about the task. Try to be sure 
they understand the task. 

• Remind them about using the yellow paper to help them get started. 

• When you feel they understand what they are supposed to do, please say, "You 
may now begin. You have 25 minutes to complete the task" and start timing 
the 25 minute period from that moment. 

At the end of 25 minutes, please collect the packets. You may discard the yellow 
sheets of paper. Write your school, teacher, name, and class in the space provided on 
the cover sheet of each student's packet. If you teach only one Galaxy class, put in 
"A." For each additional Galaxy class that you teach, use "B," "C," "D," etc. 

Hand these packets to your Data Coordinator. 

Thank you very much! 
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Writing Activity 



For Teacher to Fill Out: 

School: Teacher: Class: 

For Student to Fill Out: 

Student Name: 

(first) (last) 

Age: Birthdate: Boy Girl 

month day year (circle one) 



Instructions 

You will have 25 minutes to write a letter. Read the 
assignment carefully and think about it before you 
begin. Be sure to respond to every part of the 
assignment. Remember that you can use the 
planning page to make notes and organize your 
ideas. 

Write your letter on the lined pages. Make your 
response as thoughtful and complete as possible. 
When you are writing your letter, be sure that your 
handwriting is clear. 

If you finish before time is called, you should go 
over your work again and change anything that you 
think will make your writing better. 
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Text of Second Writing Prompt 



Writing a Letter 



For a long time, people who live near a lake called 
Loch Ness in Scotland have reported seeing a huge, 
strange animal swimming in the lake. The people 
said ^at the animal looks like a dinosaur, and they 
have tried to take pictures of it or catch it. The 
animal has been nicknamed Nessie. 

Finally, a group of scientists has figured out how to 
catch Nessie. They want to keep the animal in their 
laboratory to study it. Some people have said that it 
is unfair to take the animal out of its natural home. 

Write a letter to the scientist in charge of the project. 
Tell what you think should be done when they 
catch Nessie. Be sure to give reasons to support 
your ideas. 
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Appendix E: Performance Assessment 1 







PERFORMANCE ASSESSMENT TASK 1 
TIIEME2 






ESTABUSBOD^G YOUR OWN SPACE 
THE SECRET ROOM 




1 




Purpose 



The purpose of the writing task in this performance assessment is to allow each student to state: 

• how he/she perceives the situation in the Secret Room; 

• what position the student would take on the situation and reasons for that position 
(problem/solution); and 

• what suggestions she/he would offer to Jaimelle, Carl, and Joey (seeing others’ point of 

view). 



What Is Being Assessed 



An aspect of taking a stand (identifying a problem/proposing a solution) 
Communicating in writing 




Materials for the Teacher fo Provide 



Videotape of the A-Show to review (made during initial transmission) 
Markers for students to use in small group work 



Materials Packet 



Fax, copy for each student (with extras as needed) 
10 solution charts to be filled out in small groups 
Stationery with cover sheet for individual letters 



Copyright ® 1993 by Educational Testing Service. All rights reserved. 
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Pioc^ures 



Before Administering the Task 



1 . 

2 . 

3. 



PayJ. 

1 . 



Prior to Day 1 activities, be sure you have read the letter of introduction from the staff at 
Far West Lab (FWL). 

Decide the composition of the working groups for the small group activity on Day 1; the 
criterion may be existing groups of students, teacher choice, or student choice; the 
teacher should decide who would be a reliable scribe for each group. 

Cue the tape of the A-Show as follows so that students can review relevant portions; 
set the video counter to 00 at the beginning of the opening rap and cue it to point 01 30 
for the first segment (see below for other segments). 

Have the solution charts and markers ready for each group of students. 

Encourage students to take notes during the videotape or group or class discussions, if 
they would like, in preparation for the l^er writing activity. 



Reviewing the A-Show (approximately 8 minutes) 

• Tell students to pay particular attention to the problems that are developing between 
Jaimelle, Carl, and Joey in the Secret Room; show the following segments: 



1st segment; 01 30 to 03 90 

2nd segment: 06 75 to 07 78 
3rd segment: 1 1 03 to end. 



Beginning scene; Jannelle goes upstairs. 

Closing scene: Jannelle tells Mira she wants to be 
a writer or an actress. 

Beginning scene: Jaimelle hides her diary. 

Closing scene: Jannelle goes downstairs. 
Beginning scene: Jannelle goes upstairs. 



2. Sharing the Fax from FWL (3-5 minutes) 

• Have students form their small groups. 

• Distribute the fax from FWL. 

• Read the fax orally as students follow along. 

• Elicit from the students a few of the problems they saw developing. 

• Elicit several suggestions for possible solutions to the problems that the children are 
experiencing. 

3. Small Group Activity (15 to 18 minutes) 

• Distribute the solution charts and markers. 

• Explain that the students are to brainstorm suggestions for solutions to the problems in 
the Secret Room and are to discuss the advantages and disadvantages of tlieir solutions; 
emphasize that the brainstorming is a prewriting activity to help students prepare to 
write their letters on the next day. 

• Demonstrate on the board the way in which the scribe should fill out the solution chart. 
Use one of the solutions offered and elicit student comments about the advantages and 
disadvantages for that solution. 

(continued on page 3) 
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• Remind students that there are many different solutions and all opinions should be 
respected. 

• As students brainstorm ideas, the scribe should write brief phrases to record the ideas 
in the appropriate columns. 

• Circulate among the small groups as students work. 

• The solution charts should be taped to the wall for reference the next day to remind 
students of their discussions. 

4. Class Discussion (10 minutes at most) 

• Lead the class in a discussion to summarize the small group work. 

• Ask a volunteer from each group to suggest a solution for the problems and cite an 
advantage and a disadvantage; remind t' em that their solutions should be practical and 
fair for all three children. 

• Keep this discussion quick-paced and nonjudgmental. 

• Record their ideas on the board or an overhead transparency so that students may refer 
to them on Day 2. 

• Four or five reasonable solutions should have been presented by the end of the 
discussion. Elicit additional suggestions as needed. 

Day 2 

1 . Brief Review and Assignment (3-S minutes) 

• Very briefly recap the suggested solutions, advantages, and disadvantages from the 
previous day’s class discussion. 

• Quickly reread the fax with the students; remind the students to refer to the groups’ 
charts, the class chart, and any notes that they took as they begin to write their letters. 

• Pass out the stationwy and point out the reminders on the cover sheet of what the 
students are to include in their l^ers; also remind them to fill in the information 
requested at the top of the first sheet. 

• Tell the students that they will have only about 35 minutes to think up their letter and 
compose a first draft; pass out scratch paper if students want to make some notes 
before beginning to write. 

• Emphasize that the letters will be evaluated on how good the students’ thinkinj about 
the situation is and not on their spelling or mechanics. 

2. Individual Letter Writing (35 minutes) 

• Students write a letter to FWL. 













To Be Distributed by Teacher 






Fax from FWL 
Stationery with cover sheet 
Solution charu and markers 
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Text of Fax for Performance Assessment 1 



Dear Galaxy Student, 

We've been watching the Galaxy broadcasts along with you. We would like to find out 
what you think about what is happening as THE HOUSE. 

This week Jannelle, Joey, and Carl are having problems sharing the Secret Room. We are 
wondering if they will find a way to solve their problems. 

What do you think is the best way for the kids at the house to work things out? Please 
write a letter to us explaining your solution and tell why it's best for the kids at THE 
HOUSE. Try to convince us that your idea is a good one. You may want to include some 
of your own experiences about sharing space with others at home or at school. 

Before you write your letter, put your heads together with others from your class and 
see what their ideas are. 

Thanks for your help! 

Andrea, Cliff, Gloria, and Sue 
Far West Laboratory, San Francisco 
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STUDENT NAME 

TEACHER 

GRADE 

SCHOOL 

DATE 

INSTRUCTIONS 

Use these pages to write to us at Far West Lab. 

1. Write a sentence or two about what has 
happened so far. 

2. Tell what you think the BIG problem is. 

3. Make suggestions about how Jannelle, Joey, 
and Carl can solve their problems. 

4. Be sure to give REASONS for your 
suggestions. 
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Before Task 2 is administered: 



3. 

4. 



Decide the composition of the working groups for the small group activity on Day 1 ; the 
criterion may be existing groups of students, teacher choice, or student choice. 

Make sure that you have a full class set of the Galaxy magazine The Second Story . 
Volume 1, Issue 3; they will be reading "The True Story of a 12 year-old Hero." OR If 
you have sent the magazine home with your students, please use the master copy of the 
story provided in your materials packet to make copies for the students in your class. 
Have students sitting in their working groups before the ta^ begins. 

Write HEROIC CHARACTEF jTICS as a heading on the board or on chart paper; if the 
chart is written on the board, be sure that it can be kept intact until Day 2 or the activity. 



Day 1 



Class Discussion (15 minutes) 

• Introduce the lesson by saying that students are going to consider characterisUcs of 
individuals who can be considered heroes. Their work will consist of class and small 
group discussion, preparation of drawings, and writing. (See the last page of these 
instructions for options about displaying the drawings.) 

• Have students turn to "The True Story of a 12 year-old Hero" on the last page of the 
magazine or hand out the copies you have made. 

• Read "The True Story" orally as students follow along; this reading should not take 
long. 

• Lead the class in a quick-paced discussion of the cliaracteristics of people who can be 
considered heroic. Students may mention the characteristics depicted in the story from 
The Second Storv or characteristics of other individuals; they should be encouraged to 
draw upon their own ideas and upon literature they have read. Be sure students 
mention or acknowledge diverse characteristics of heroes that include (but are not 
limited to) those listed: 

• Ordinary people can be heroic. 

• Heroic people can be any age. 

• Both females and males can be heroic. 

• Heroes can have weakness as well as strengths. 

• Heroes may not always succeed at everything they try to do. 

• There are disadvantages to being considered a hero. 

• As the students make suggestions, summarize their ideas by listing brief descriptive 
phrases on the HEROIC CHARACTERISTICS chart. Help the students recognize the 
following: 

• The usual concept of “hero" from sports or entertainment is different from the 
idea being discussed here; people from these fields may become heroes, but 
their identities as "stars" do not immediately make them heroic. 

• There is a difference between being a hero and being a "helper"; this concept 
will be developed further in Task 3 (Theme 6). 



(continued on page 3) 
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• Stress that people usually do not plan to become heroic; often, a person becomes a 
hero by responding to extraordinary circumstances (that are often dangerous) in 
highly extraordinary ways (that often put the person "on the line" in some way); in 
other cases, individuals live their lives in heroic ways, working for the causes and 
issues that they consider important and often overcoming social, political, physical, 
or other obstacles. 

2. Small Group Activity (5-7 minutes) 

• Tell the students that they are to spend the next few minutes extending the class 
discussion; this small group work will help students affirm their individual concepts 
about heroes. 

• State that each group member is to think of people whom they consider to be heroes 
and that they will have to select one of those people to depia in a drawing and to 
discuss further in writing. In their discussion, they are to provide evidence to support 
their choice by citing charaaeristics aixl behaviors that have made these people heroes; 
this process should help students identify the person ab^>u» whom they will write. 

3. Preparation of Drawings (10 minutes) 

• Tell the students to begin drawing about one person (young or old, female or male) 
whom they consider a hero; they may draw a portrait or a scene depicting some aspect 
of the person’s life; they should also write a few phrases that capture why their 
selection can be considered a hero, referring if necessary to the chart developed by the 
teacher. 

(Note: While the drawings may be displayed in the classroom at the end of the 
aaivity, stress that now students should consider their drawing a prewriting 
exercise; if they do not finish their drawings within the time period, they may finish 
them later; if they are reluctant to draw ("I’m not a good artist"; "I don’t know 
what the person really looked like," etc.), reassure them that they will not be 
"graded" on their artistic ability or accuracy.) 

• Circulate among the small groups as students work, providing help and encouragement; 
make sure that each student has written the name of the hero on the drawing. 

• After 10 minutes, call time again and ask students to share their hero and their 
thoughts about what makes this person heroic with their group; make sure they realize 
that their efforts so far may still be works-in-progress. 

Pay,.2 

1 . Brief Review and Assignment (10 minutes) 

• Read the fax from FWL and remind students that Andrea, Cliff, Gloria, and Sue are 
their audience. 

• Briefly review the HEROIC CHARACTERISTIC chart, explaining that this material 
may be helpful to students as they write. 

(continued on page 4) , 
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• Explain to students that they are to write about the person they selected on the previous 
day and are to provide evidence for their choice. 

[If students ask, tell them that they may change their choice of hero from the 
previous day; it is possible students will have thought more about whom they 
consider a hero and have new ideas.] 

2. Individual Writing Assignment (20 minutes) 

• Tell students to compose a first draft of the statement/letter to FWL supporting their 
choice. 





Options for Using the Drawings 



• Arrange the drawings on a bulletin board; be sure that each drawing is labeled with the 
name of the hero it depicts. 

• Teachers may also want to cooside.” students’ drawings a first draft of work to be 
revised by students and used to construct a "hero quilt" (as discussed on page 2 of the 
green "Sample Activities for Theme 3" supplied by Galaxy). 

• Alternatively, the drawings could be revised and then published in a book for the class 

library. 
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Text of Fax for Performance Assessment 2 



Dear Galaxy Student, 

Hello again! Thanks for your letters about the problem at THE HOUSE. You all had 
some great ideas for how Jaimelle, Joey, and Carl should share the secret room. 

We know that you have been talking, reading, writing and drawing about heroes lately, 
and we would like to hear about your own personal heroes. Please send us a descrip- 
tion of your own hero and try to convince us why you think he or she has been heroic. 

We're looking forward to reading about your heroes! 

Andrea, Sue, Gloria and Cliff 
Far West Laboratory 
San Francisco, California 
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STUDENT NAME 

TEACHER 

GRADE 

SCHOOL 

DATE 

INSTRUCTIONS 

Use these pages to write to us at Far West Lab 
giving your description of a person you consider to 
be a hero. Make sure that you: 

1. Give die name of the person you have 
selected as a hero. 

2. Explain why you think this person is a hero. 

3. Give REASONS to convince Andrea, Sue, 
Gloria, and Cliff that your person is truly a 
hero. 
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Appendix G: Performance Assessment 3 



PERFORMANCE ASSESSMENT TASK 3 
THEME 6 

PEOPLE NEED PEOPLE 



^ Purpose 

The purpose of this performance assessment is to allow each student to: 

• state what he/she believes to be the characteristics of people who are helpers 

• provide examples of actions that are helpful 

• develop a drawing depicting an individual whom the student considers to be a helper 

• prepare a written discussion that cites evidence to support the student's selection 



What Is Being Assessed 


! 


Taking a stand/presenting a position and offering supporting evidence 
Communicating in writing 


1 



Time Frame. 



Between A*Show and B*Show 

Two consecutive school days with class periods of approximately 30 minutes each 



1 Paper axKi drawing materials 


□ 






Materials Packet 




1 Sutionery with cover sheet for student’s written work | 

H Fax from Far West Lab (FWL) 1 


B Copy of story about Trevor Ferrell 





Copyright * 1993 by Galaxy Institute for Education. All rights reserved. 



O 



-13 



113 



Language Arts Final Report 




Before Task 3 is administered: 



1 . 

2 . 



3. 

4. 



Decide the composition of the working groups for the small group activity on Day 1; the 
criterion may be existing groups of students, teacher choice, or student choice. 

Make sure that all your students have their copy of the story about Trevor Ferrell; use the 
copy of the story provided in your materials packet to make any extra copies that may be 
needed. 

Have students sitting with their working groups before the task begins. 

Write CHARACTERISTICS AND ACTIONS OF PEOPLE WHO ARE HELPERS as a 
heading on the board or on chart paper; if the chart is written on the board, be sure that it 
can be kept intact until Day 2 of ^e activity. ^ 



Day I 

1. Class Discussion (15 minutes) 

• Introduce the lesson by telling students that they are going to consider characteristics 
and actions of individuals who can be consider^ helpers. Their work will consist of 
class and small group discussions, preparation of drawings, and writing. (See the last 
page of these instructions for options about displaying the drawings.) 

• Read the story about Trevor Ferrell orally, as the students follow along silently on 
their own copies. 

• Lead the class in a quick-paced discussion of the characteristics and actions of people 
who can be considered helpers. Students may mention the characteristics and actions 
depicted in the story of Trevor Fenell or characteristics of other individiials. Students 
should be encouraged to draw upon their own ideas and experiences, am. upon 
literature they have read. Be sure students mention or acknowledge diverse 
characteristics and behaviors , such as: 

• being kind, friendly, considerate 

• running errands 

• visiting people who cannot leave home 

• volunteering to baby-sit 

• As students discuss, summarize their ideas to key phrases; enter these phrases in the 
chart of CHARACTERISTICS AND ACTIONS OF PEOPLE WHO ARE HELPERS. 
In the discussion, students may mention characteristics or actions; in all probability, 
students will mention both aspects of helpers. Develop the point that people who are 
helpers may have some heroic characteristics (as developed in Task 2) but that their 
behaviors are usually ongoing, long-term, and arising from a particular inclination to 
be helpful; discussion should include, but not be limited to, the following points: 

• Helpers can be any age. 

• Both females and males can be helpers. 

• Sometimes, behaviors that are intended to be helpful may not be perceived 
accurately. 

• There may be disadvantages to being considered a helper; people may take 
advantage of helpers. 



I (con 



(continued on page 3) 



J 
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• Stress that people usually do not plan to become helpers, any more Aan they plan to 
become heroes; also, being a helper does not have to be a full-time job, although 
some people, like Trevor Ferrell, find that their first helping action turns into a 
large and important effort; students may also suggest that nurses and teachers are 
helpers. 

2. Small Group Activity; Brainstorming (5-7 minutes) 

• Tell the students that they are to sperd the next few minutes extending the class 
discussion; this small group work will help students affirm their individual concepts 

about helpers. 1 

• State that each group member is to think of people whom they consider to be helpers | 
and that they will have to select one of those people to depict in a drawing and to | 
discuss further in writing. In their discussion, they are to provide evidence to support | 
their choice by citing characteristics and actions that have made these people helpers; 
this process should help students identify the flnfi person about whom they will write. 

The people whom they discuss may be individuals they know personally or people they 
have heard or read about; encourage them to extend their thinking to people other than 
those in their immediate family or community but do not devalue their ideas if they 
insist upon writing about someone close to them. [It might be a good idea to | 

discourage the students from writing about their teacher, however!] 

3. Individual Preparation of Drawings (10 minutes) 

• Tell the students to begin drawing a picture about onfi person (young or old, female or 
male) whom they consider a helper; they may draw a portrait or a scene depicting how 
the person helps others; they should also write a few phrases that capture why their 
selection can be considered a helper, referring if nei cssary to the chart developed by 
the class and transcribed by the teacher. 

[Note; While the drawings may be displayed in the classroom at the end of the 
aa.' dty, stress that now students should consider their drawing a prewriting 
exercise: if they do not finish their drawings within the time period, they may finish 
them later; if they are reluctant to draw ("I’m not a good artist"; "I don’t know 
what the person really looked like," etc.), reassure them that they wUl not be 
"graded" on their artistic ability or accuracy.] 

• Circulate among the small groups as students work, providing help and encouragement. 
Make sure that each student has written the name of the helper on the drawing and a 
few phrases to describe why that person is a helper. 

• After 10 minutes, call time again and ask students to share ideas about their helper and 
their thoughts about what makes this person helpful with their group; make sure they 
realize that their efforts so far may still be works-in-progress. 

Pay.2 

1. Brief Review and Assignment (10 minutes) 

• Read the fax from FWL and remind students that Andrea, Cliff, Gloria, and Sue are 
their audience. 

• Review the CHARACTERISTICS AND ACTIONS OF PEOPLE WHO ARE 

HELPERS chart, explaining that this material may be helpful to students as they wri^ 
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• Explain to students that they are to write about the person they selected on the previous 
day and are to provide evidence for their choice. 

[If students ask, tell them that they may change their choice of helper from the 
previous day; it is possible students will have thought more about whom they 
consider a helper and have new ideas.] 

2. Individual Writing Assignment (20 minutes) 

• Tell students to compose a first draft of the statement/letter to FWL supporting their 
choice. 



To Be Distributed ty Tca^^ 


= ^ 


Stationery with cover sheet 
Story about Trevor Ferrell 


i 





What Is Tumed;.In 


! 


1 Individual written work 









Arrange the drawings on a bulletin board; be sure that each drawing is labeled with the 
name of the helper it d^icts. 

Teachers may want to consider students’ drawings a first draft of work to be revised 
by students and used to construct a "helper quilt" (as discussed on page 2 of the green 
"Sample Activities for Theme 6" supplied by Galaxy). 

Alternatively, the drawings could be revised and then published in a book for the class 
library. 



Copyright ® 1993 by Galaxy Institute for Education. All rights reserved. 



4 




lie 



Appendices 



BEST COPY AVAILABLE 



116 



Text of Fax for Performance Assessment 3 



Dear Students, 

Hello again! It was great to read about your heroes. You all had some good reasons for 
choosing them. 

We know that you have been talking, reading, writing and drawing about helpers lately, 
and we would like to hear about people you think are helpers. Please send us a descrip- 
tion of the helper you have chosen to write about. Tell us about what this person does or 
has done and convince us that the person is really a helper. 

We're looking forward to reading about the people you think are helpful and about how 
they help other people! 

Your Friends, 

Andrea, Sue, Gloria and Cliff 
Far West Laboratory 
San Francisco, California 
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STUDENT NAME 

TEACHER 

GRADE 

SCHOOL 

DATE 

INSTRUCTIONS 

Use these pages to write to us at Far West Lab 
giving your description of a person you consider to 
be a helper. Make sure that you: 

1. Give the name of the person you have 
selected as a helper. 

2. Explain why you think this person is a helper. 

3. Give REASONS to convince Andrea, Sue, 
Gloria, and Cliff that your person is truly a 
helper. 
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Appendix H: Performance Assessment 4 



PERFORMANCE ASSESSMENT TASK 4 
THEME? 

PEOPLE WORKING TOGETHER CAN MAKE A DIFFERENCE 




Puipose 



The piupose of the writing task in this perfomuaice assessment is to allow each student to 
state: 

• how he/she perceives the situation among Carl, Miguel, and Joey; 

• what position the student would take on the situation and reasons for that 
position (problem/solution); and 

• what suggestions she/he would offer to Carl, Miguel, and Joey (seeing others’ 
point of view). 




What Is Being Assessed 



An aspect of taking a stand (identifying a problem/pioposing and supporting a 
solution) 

Communicating in writing 



Tiine Frame 



■ 



Between the A>Show and the B-Show 

Two consecutive school di^ with class periods of approximately 30-35 minutes each 



Matmials for the Teacher to Provitfe 



\^deotape of the A-Show to review (make sure to record the A-Show during initial 
transmission) 

Markets for students to use in small group worit 

Scratch paper for students to uke notes or organize their ideas before writing 



Copyright* 1993 by Galaxy Institute for Education, All rights reserved. 



U£ 



" BEST COPY AVAILABLE 



Language Arts Final Report 



Materials Packet 


Pax, copy for each student (with extras as needed) 
10 solution charts to be filled out in small groups 
Stationery with cover sheet for individual letters 
















1 Procedures | 



Before Adnunistering the Task 



1. Decide the composition of the working groups for the small group activity on Day 1; 
the criterion may be exLtting groups of students, teacher choice, or student choice; the 
teacher should decide who would be a reliable scribe for each group. 

2. Cue or fast forward th/i tape of the A-Show to immediately after the opening rap so 
that students can review the show. 

3. Have the solution charts and maricen ready for each group of students. 

4. Encourage students to take notes during the videotape or during group or class 
discmsion in preparation for the letter-writing activity. 

Pay.1 

1. Reviewing the A-Show (approximately 10-15 minutes) 

• Tell students that as they review the A-Show, they should pay particular atteruion to 

the problems developing among Miguel, Cari, and Joey with regard to their act for 

the talent show. 

2. Sharing die Fax from FWL (3-5 minutes) 

• Have students form their small groups. 

• Distribute the frx from FWL. 

• Read the fax orally as students follow along. 

• Elicit from the students the problems they see developing among Miguel, Joey, 
and CarL 

• Elicit several suggesdoiu for possible solutiom to the problems that the boys are 
experiencing. 

3. Small Giotqi Acdvi^ (15-18 minutes) 

• Distribute the soludon charts and maricers. 

• Eiqilain that the students are to brainstorm suggestions for solutioiu to the 
problems among Joqr, Miguel, and Cari and ate to discuss the advantages and 
disadvantages of thefr solutions; emphasize that the brainstorming is a prewridng 
acdviQr to help students prepare to write their letters on the next day. 

• Demonstrate on the board the way in which the scribe should fill out the solution 
chart Use one of the solutions offered and elicit student comments about the 
advantages and disadvantages for that solution. 

(continued on page 3) 
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Remind students that there are many different solutions and that all opinions 
shotild be respected. 

As students brainstorm ideas, the scribe should write brief phrases to record the 
ideas in the appropriate columns. 

Circulate among the small groups as students work. 

The solution charts shotild be taped to the wall for reference the next day to 
remind students of their discussions. 



4. Class Discussbn (10 minutes at most) 

• Lead the class in a discussion to summarize the small group work. 

• Ask a volunteer from each group to suggest a solution tor the problems and 
dte an advantage and a disadvantage of the solution; remind them that their 
solutions should be practical and fair for all three chQdrem 

• Keep this discussion quick-paced and nonjudgmentaL 

• Record their ideas on the board or an overhead transparency so that students 
may refer to them on Day 2. 

• Four or five reasonable solutions should have been presented by the end of the 
discussion. Elicit addidorud suggesdons as needed. 



P a y .2 



1. Brief Review and Assignment (3-5 minutes) 

• Very briefly recap the suggested solutions, advantages, and disadvantages from 
the previous day’s class discussioxL 

• Quickly reread the fax with the students; remind the students to refer to the 
group^ charts, the class chart on the board or overiiead transparency, and any 
notes that they took as they begin to write their letters. 

• Pass out the stadonery and point out the reminders on the cover sheet of what 
the students are to indude in their letters; also remirui them to fill in the 
informadon requested at the top of the first sheet if there is no label on their 
fint sheet 

• Tell the students that they will have only about 35 minutes to think up their 
letter and conqrose a fint draft; pass out scratch paper if students want to 
make some notes or organize their ideas before beginning to write. 

• Emphasize that the letten vdll be evaluated on how good the students’ thinking 
about the situation is and not on their spelling or mechanics. 

2. Individual Letter Writing (35 minutes) 

• Students write a letter to FWL 
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Text of Fax for Performance Assessment 4 



Dear Galaxy Student, 

Wow! We can't believe that the school year is almost over and we'll be saying good-bye 
to Galaxy for a while. But before the year is over, we would like to find out one last time 
what you think about what is happening at THE HOUSE. 

As you know, Miguel, Carl, and Joey are having problems working together on the 
talent show. In fact, they aren't working together an)ntnore and their problems may 
affect the success of the show. 

You have had such good ideas about how the kids at THE HOUSE could solve the other 
problems they have faced. What do you think is the best way for Miguel, Carl, and Joey 
to work things out? 

Write a letter to us explaining your solution and tell why it's the best solution for the 
kids at THE HOUSE. Try to convince us that your idea is a good one. You may want to 
include some of your own experiences about working with others. 

Before you write your letter, put your heads together with others from your class and 
see what their ideas are. 

Thanks for your help! 

Andrea, Cliff, Gloria, and Sue 
Far West Laboratory, San Francisco 
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STUDENT NAME 

TEACHER 

GRADE 

SCHOOL 

DATE 

INSTRUCTIONS 

Use these pages to write to us at Far West Lab 
about your solutions to the problem. 



1. Write a sentence or two about what has 
happened so far with the talent show. 

2. Tell what you think the BIG problem is. 

3. Make suggestions about how Miguel, Joey, 
and Carl can solve their problems can solve 
their problems. 

4. Be sure to give REASONS for your 
suggestions. 
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Appendix I. Performance Assessment Scoring Guides 



SCORING GUIDE: Task 1 



Scoring Rationale: The directions requite that the respondent take a stand by suggesting a 

solution to the problem Jannelle, Joey, and Carl are having sharing the Secret Room, and to 

provide reasons to support the solution as a good one. 

5 OUTSTANDING RESPONSE 

* Students present a statement that indicates they have taken a stand or that they have 
proposed a compromise by suggesting a solution to the problem. 

• Students present an elaborated, well-developed set of con^lling, logical, and 
thoughtful reasons to support their stand. Students indicate an awareness of multiple 
perspectives. As evidence to support their stand, sti^dents draw upon references from 
the stories at THE HOUSE, from material they have read or seen elsewhere, or from 
their own experience. 

* It is clear from the students' woric that they undentand the broader issues, such as the 
need to respect otiieis' tight to privacy ano/or the need to share space. 

4 COMPETENT RESPONSE 

• Students present a statement that indicates they have taken a stand or that they have 
proposed a compromise by suggesting a solution to the problem. 

• Students present reasons to suppon their stand but these may be less corralling, 
lo^cal, thoughtful, or diverse than in responses ranked higher. Students suggest there 
are multiple perspectives to the situation. The discussion may be unevenly OTganized 
but students demonstrate some development of their position. 

3 SUFFICIENT RESPONSE 

* Students present a statement that indicates they have taken a stand or that drey have 
proposed a compromise by suggesting a solution to the problem. 

• Students provide minimal evidence to support their solution or recommendations, with 
litde specific detail Responses in this category typically do not include multiple 
persprctives. Ihe discussion is not well organist and/or developed. 

2 MINIMAL RESPONSE 

• Students state or imply a stand about the situation. They may offer suggestions about 
what should have bm done to prevent the initial problem or assign blame, rather than 
suggesting a solution. For example, they may surest that Jannelle should have taken 
her diary with her when she left the room, or that Mira should not have told the boys 
about this diary. 

* Students do not offer support for their positions. The discussion is undeveloped. 

1 INCOMPLETE RESPONSE 

* Students attenqrt to respond to the topic but do not seem to have an undetstatMling of 
the ta^ and/or do not understand the problem presented in the videot^. Studetus 
respond to the topic in very general terms. For example, they may refer to getting along 
or needing to help each other. Alternately, they may restate the t^s or retell the events 
presented in the video. 



0 UNSCORABLE 

• Students do not respond in writing, the writing is illegible, or students may not have 
expressed a thou^t complete enough to score as a 1. 






best copy available 125 



Language Arts Final Report 



SCORING GUIDE: Task 2 



Scoring Rationale: The directions require that the respondent identify someone that is a hero 
and provide reasons for the choice. Writing mechanics are not to be considered when scoring. 

5 OUTSTANDING RESPONSE 

• Students identify a person whom they consider a hero; he/she may be from real life or 
horn literature. 

• Students give compelling, logical, and thoughtful reasons to support their choice and 
provide ample evidence about the person or his/her behavi<»s to substantiate their 
contention that the person has been heroic. 

• It is clear fitom the students' work that they have selected an individual who has 
perfonned in a heroic manner at some point or who has continued to perform heroically 
in supptm of a cause or issues; students do not offer 3ports or entertainment 
personalities without specific luid accurate reference to behavior that qu^fiies to make 
the person heroic; students have not confused heroic behavior with hdping betuvior or 
"stardom." 

4 COMPETENT RESPONSE 

• Students identify a person whom they consider a hero; he/she may be from real life or 
from literature. 

• Students present reasons to suppm their choice. They ^vide sufficient evidence 
about the person or his/her behaviors to substantiate thetr contention that the person has 
been heroic, but these may be less compelling, logical, or thou^tful than in responses 
ranked higher; the discussion is less ad^uately developed and ori^ized. 



3 SUFFICIENT RESPONSE 

• Students identify a hero but provide minimal evidence, with little specific detail about 
the person's life or behaviors to support their choice. 

• The discussion is not well-organized and/or developed and may indicate some 
confusion about the characteristics of a hero. There may be confusion between heroic 
and helping behaviors; the person identified as a hero or the reasons for the selection 
may be inappropriate or unclear. 

2 MINIMAL RESPONSE 

• The discussion is poorly developed. Students identify a hero and attempt to support 
their position but die evidence is general and vague. Students may attempt to support 
their position by listing heroic attributes without any supporting examples. There may 
be confusion between heroic and helping behavkus. 

1 INCOMPLETE RESPONSE 

• Students may or may not identify a hero; they may sii^ly state a name. They do not 
seem to understand the task and/or the concept of heroic behavior. They do not offer 
supporting evidence for their position. 

0 UNSCORABLE RESPONSE 

• Students do not respond in writing, the writing is illegible, or students may not have 
expressed a thought complete enough to score as a 1. 
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SCORING GUIDE: Task 3 



Scoring Rationale: The directions require that the respondent identify someone that is a helper 
and provide reasons fen* their choice. Writing mechanics are not to be considered when scoring. 

5 OUTSTANDING RESPONSE 

• Students identify a person whom they consider a helper, he/she may be hx>m real life or 
from literature. 

• Students g?ve compelling, logical, and thoughtful reasons to support their choice and 
provide ample evidence about the person or his/her behaviors to substantiate their 
contention that the person is helper. 

• It is clear from the students' work that th^ have selected an individual who has 
demonstrated characteristics and/or actions of people who are helpers. People who are 
helpers may have some heroic characteristics, but their behaviors are usually ongoing, 
long-term, and arising team a particular incliiiation to be helpful. 

4 COMPETENT RESPONSE 

• Students identify a person whom they corudder a helper; he/she may be someone they 
know personally or about whom they have read or heanl. 

• Students present reasons to support their choice. They provide sufficient evidence 
about the person or his/her behaviors to substantiate their contention that the person is a 
helper, but these may be less compelling, logical, or thoughtful than in responses 
ranked higher; the discussion is less adequately developed and organized. 



3 SUFFICIENT RESPONSE 

• Students identify a helper but provide minimal evidence, with little ^lecific detail about 
the person's life or behaviors to support their choice. 

• The discussion is not well-organized and/or developed and may indicate some 
confusion about the characteristics of a helper. There may be confusion between heroic 
and helping behaviors; the person identified as a helper or the reasons for the selection 
may be inappropriate or unclear. 

2 MINIMAL RESPONSE 

• The discussion is poorly developed. Students identify a helper and attempt to support 
^eir position but Ae evidence is general and vague. Students may attem]^ to support 
their position by listing attributes of a helpn without any supporting examples. There 
may be confusion between heroic and helping behaviors. 

1 INCOMPLETE RESPONSE 

• Students may OT may not identify a helper, they may simply state a name. They do not 
seem to undmtand the task and/or the concept of the chanu;tcristics and actions of 
people who are helpers. They do not offer supporting evidence for their position. 

0 UNSCORABLE RESPONSE 

• Students do not respond in writing, the writing is illegible, or students may not have 
expressed a thought complete enough to score as a 1. 
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SCORING GUIDE: Task 4 



Scoring Rationale: The directions require that the respondent take a stand by suggesting a 

solution to the problem Joey, Cari, and Miguel arc having waddng as a team for the talent show 

and to provide reasons to support the solution as a good one. 

S OUTSTANDING RESPONSE 

• Students present a statement that indicates they have taken a stand or that they have 
proposed a compromise by suggesting a solution to the problem. 

• Students present an elaborated, well-developed set of compelling, logical, and 
thoughtful reasons to support their stand. Students indicare an awareness of multiple 
perspectives. As evidence to support their stand, students draw upon references from 
the stories at THE HOUSE, from material they have read or seen elsewhere, or from 
their own experience. 

• It is clear from the students' work that they understand the broader issues, that 
co(^>eration and teamwork are difficult yet necessary for the good of the show. 

4 COMPETENT RESPONSE 

• Students present a statement that indicates they have taken a stand or that they have 
proposed a compromise by suggesting a solution to the problem. 

• Students present reasons to support their stand but these may be less compelling, 
logical, Aoughtful, or diverse than in response ranked higher. Students suggest there 
are multiple perspectives re the situation. The discussion may be unevenly orgamzed 
but students demonstrate some develqrment of their position. 

3 SUFFICIENT RESPONSE 

• Students present a statement that indicates they have taken a stand or that they have 
proposed a compronnise by suggesting a solution to the problem. 

• S tudents provide minimal evideiree to suppwt their solution or recommendations with 
little specific detail. Respmses in this category typically do not include multiple 
perspectives. The discussion is not well organiz^ and/or develop^ 

2 MINIMAL RESPONSE 

• Students state or imply a stand about the situation. They may offer suggestions about 
what should have been done to prevent the initial problem or they may assign blame, 
rather than suggesting a solution. Forexample, they may suggest that Cari should 
have asked Migud before including Joey or that Joey should not have quit the tearrL 

• Students do irot offer support for their positions. The discussion is undeveloped. 

1 INCOMPLETE RESPONSE 

• Students attempt to respond to the topic but do not seem to have an understanding of the 
task and/OT ^ not understand the problem presented in the videoupe. Students respond 
to the topic in very general terms. For example, they may refer to getting along with 
others or needing to help each other. Alternately, they may restate the tTdc* or retell the 
events presented in the video. 

0 UNSCORABLE RESPONSE 

• Students do not respond in writing, the writing is illegible, or students may not have 
expressed a thought complete enough to score as a 1. 
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Appendix J. Writing Prompt Scoring Guides 



Scoring Guide: Writing Prompt #1 



Spaceship OlWll 

Books W67, W68, W73, W77 Grade 4 

Persuasive 

SCORING GUIDE 
Primary Trait 

Primary trait : Persuasion through articulation and support of a position. 

Scoring rationale : The directive requires respondents to take a position about whethw the spaceship 
should be allowed to return to its planet, and to provide reasons to supptm their position. 



6 Extensively elaborated. In these responses, students articulate a position for or against the 
return of the spaceship, or suggest a plan, and they present an extended, well-written 
discussion on the reasons for their position. These responses may be similar to "5" responses, 
but they are better organized, more clearly written, and less flawed. 

5 Elaborated. In these responses, students aniculate a position for or against the return of the 
spaceship, or suggest a plan, and they provide an extended discussion on the reasons for 
their position. Or they present an argument based on a strong sense of audience. 

4 Developed. In these responses, students uke a position for or against the return of the 
spaceship, or suggest a plan, and they discuss the reasons or give a brief argument for their 
position. Although the reasons may be more compelling or more clearly stated titan in papers 
that received lower scores, the discussion may be unevenly developed. 

3 Minimally developed. In these responses, students sute or imply a position for or against 
the return of the spaceship and give at least one substantive reason or seve^ brief reasons to 
support their stand. Or, they present a plan for resolving the problem that includes seme 
explanation about its tenefits. These papers can be brief or luve vague and confosing aspects. 
Note: Reasons can include humanitarian appeals (e.g., the scientists would not like to be 
studied, the aliens have a right to live their own life.) 

2 Undeveloped response to task. In these responses, students state a position for or 

against the return of the spaceship or they present a plan for resolving the problem, ^t they offer 
no reasons or only vagu^nappropriate/imelevant reasons to support their point of view. (>, 
they present confusing plans. Or, they present a reason or two on each side of the issue without 
articulating that they are undecided. 

1 Response to topic. In these responses, students respond to the topic but do not appear to 
have understood the task or they may appear to state a position but the paper is largely 
incomprehensible. For example, they may refer to spaceships in general or about aliens 
visiting the earth without any indication of what to do about the problem or why. Or, they 
rccopy text from the prompt 
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SCORING GUIDE: Writing Prompt # 2 



Primary trait: Persimion through articulation and support of a position 

Scoring rationale: The directive requires respondents to take a position about what should 

be done with Nessie when caught or argue against a capture or present a compromise. 

6 Extensively elaborated. In these responses, students articulate a position about 
what should be dwie when Nessie is caught or argue against a capture or present a 
conqjromise. Students present an extended, well>written discussion on the reasons 
for their position. These responses may be similar to "5" responses, but they are 
better organized, nx)te clearly written, and less flawed. 

5 Elaborated. In these responses, students articulate a position about what should 
be done when Nessie is caught or argue against a cm>ture or present a 
compromise. Students pro^e an extended discussion on the reasons for dieir 
position. Or they present an argument based on a strong sense of audience. 

4 Developed. In these responses, students take a position about what should be 
done with Nessie when caught or argue against a capture or present a compromise. 
Students discuss the reasons or give a brief argument for theff position. Although 
the Jasons may be more coi^lling or more clearly stated than in papers that 
received lower scores, the discussion may be unevenly developed. 

3 Minimally developed. In these responses, students state or imply a position or 
cortqnomise. Students should give at least one substantive reason or several brief 
reasons to support their position. These papers can be brief or have vague and 
confusing aspects. Note: Reasons can include humanitarian appeals. 

2 Undeveloped response to task. In these responses, students state a position 
about what should be done with Nessie, but they offer no reasons or only 
vaguerinappropriaterirrelevant reasons to support their point of view. Or they 
present confusing plans. Or they present a reason or two on each side of the issue 
without articulating that they arc undecided. 

1 Response to topic In these responses, students respond to the topic but do not 
appear to have understood the task or they may appear to state a position but their 
arguments are largely incomprohensible. For example, they may refer to diiK^un 
or "monsters" in general or without any indication of what to do abwt the problem. 
Or they may recopy text from the prompt 
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Appendix K. Results From the End of Year Teacher Survey 



1. The Galaxy language arts cycle lasted 14 weeks. During approximately how many 
weeks was the following equipment not working? Please put an X in the 
appropriate box on each line. 





Never 

Worked 


1-2 Weeks j 3^4 Weeks 


5^ Weeks 

"T ■ ■ 


7 Or Mors 
r:- Weeks 


^TAIwaysJ^ 
Worked b:: 


Fax machine 


3 

(2.9%) 


26 

(25.5%) 


26 

(25.5%) 


12 

(11.8%) 


27 

(26.5%) 


8 

(7.8%) 


TV monitor 


1 

(1.0%) 


9 

(9.3%) 


1 

(1.0%) 


1 

(1.0%) 


4 

(4.1%) 


81 

(83.5%) 


VCR 


2 

(2.0%) 


4 

(4.1%) 


1 

(1.0%) 


2 

(2.0%) 


4 

(4.1%) 


85 

(86.7%) 


Voice connection 
(hoot 'n holler) 


14 

(15.7%) 


8 

(9.0%) 


3 

(3.4%) 


2 

(2.2%) 


7 

(7.9%) 


(61.8%) 



2. How often did you and/or your class use the following equipment as part of 
Galaxy? Please put an X in the appropriate box on each line. 





;^N0t.atAU^ 




^^dnce-^r-^4' 






Fax machine 


5 


19 


20 


48 


11 




(4.9%) 


(18.4%) 


(19.4%) 


(46.6%) 


(10.7%) 


TV monitor 


5 


2 


12 


73 


11 




(4.9%) 


(1.9%) 


(11.7%) 


(70.9%) 


(10.7%) 


VCR 


4 


5 


20 


68 


8 




(3.8%) 


(4.8%) 


(19.0%) 


(64.8%) 


(7.6%) 


Voice connection 


36 


46 


5 


9 


3 


(hoot n' holler) 


(36.4%) 


(46.5%) 


(5.1%) 


(9.1%) 


(3.0%) 


Galaxy computer 


68 


14 


3 


5 


6 




(70.8%) 


(14.6%) 


(3.1%) 


(5.2%) 


(6.2%) 


Other classroom 


21 


22 


11 


19 


25 


computers 


(21.4%) 


(22.4%) 


(11.2%) 


(19.4%) 


(25.5%) 


Camcorder 


12 


81 


3 


1 


1 




(12.2%) 


(82.7%) 


(3.1%) 


(1.0%) 


(1.0%) 



=102 

f=97 

f=98 

[=89 

=103 

=103 

=105 

=99 

=96 

=98 

=98 
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3. Many of the Galaxy program components are listed below. Please put an X in the 
appropriate boxes in the first column to indicate your opinion of their educational 
^ value. In the second column, indicate how well your students liked the components. 





Great 

Value 


Educatior 

Some 

Value 


ial Value 

Uttio 

Value 


Did Not 
Use 


Galaxy broadcasts 


99 

(95.2%) 


5 

(4.8%) 


0 

(0.0%) 


0 

(0.0%) 


Galaxy themes 


96 

(92.3%) 


8 

(7.7%) 


0 

(0.0%) 


0 

(0.0%) 


Galaxy activities 


89 

(87.3%) 


12 

(11.8%) 


1 

(1) 


0 

(0.0%) 


Galaxy literature 
books 


94 

(91.3%) 


8 

(7.8%) 


1 

(1.0%) 


0 

(0.0%) 


THE SECOND STORY 
(magazine) 


60 

(58.8%) 


39 

(38.2%) 


2 

(2.0%) 


1 

(1.0%) 


THE MAX 


41 

(41.8%) 


43 

(43.9%) 


7 

(7.1%) 


7 

(7.1%) 


Galaxy portfolios 


55 

(54.5%) 


36 

(35.6%) 


8 

(7.9%) 


2 

(2.0%) 


Use of fax machine 


75 

(73.5%) 


18 

(17.6%) 


4 

(3.9%) 


5 

(4.9%) 


Use of voice 
connection 


11 

(11.2%) 


26 

(27.1%) 


19 

(19.8%) 


40 

(41.7%) 


Reviewing broadcasts 
with VCR 


89 

(86.4%) 


11 

(10.7%) 


1 

(1.0%) 


2 

(1.9%) 


Galaxy partner school 
communications 


44 

(42.3%) 


27 

(26.0%) 


18 

(17.3%) 


15 

(14.4%) 


Communications with 
other Galaxy teachers 


31 

(30.4%) 


34 

(33.3%) 


10 

(9.8%) 


27 

(26.5%) 



N=104 



N=104 



N=102 



N=103 



N=102 



N=98 



N=101 



N=102 



N=96 



N=103 



N=104 



N=102 



1 Ukeda 

1 


Studen 

Uked 

Some- 

What 


ts Like 

r. 

Did Not 
Uka 


Did Not 
Use 


101 

(8.1%) 


2 

(1.9%) 


0 

(0.0%) 


0 

(0.0%) 


91 

(89.2%) 


10 

(9.8%) 


1 

(1.0%) 


0 

(0.0%) 


79 

(76.7%) 


24 

(23.3%) 


0 

(0.0%) 


0 

(0.0%) 


94 

(90.4%) 


10 

(9.6%) 


0 

(0.0%) 


0 

(0.0%) 


67 

(66.3%) 


34 

(33.7%) 


0 

(0.0%) 


0 

(0.0%) 


48 

(49.5%) 


40 

(41.2%) 


2 

(2.1%) 


7 

(7.2%) 


51 

(52.0%) 


39 

(39.8%) 


5 

(5.1%) 


3 

(3.1%) 


86 

(86.0%) 


6 

(6.0%) 


0 

(0.0%) 


8 

(8.0%) 


24 

(25.8%) 


17 

(18.3%) 


2 

(2.2%) 


50 

(53.8%) 


90 

(88.2%) 


9 

(8.8%) 


1 

(1.0%) 


2 

(2.0%) 


62 

(63.3%) 


23 

(23.5%) 


5 

(5.1%) 


8 

(8.2%) 


33 

(35.5%) 


20 

(21.5%) 


0 

(0.0%) 


40 

(43.0%) 



N=103 



N=102 



N=103 



N=104 



N=101 



N=97 



N=98 



N=100 



N=93 



N=102 



N=98 



N=104 
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4. Student use of a fax machine is one of the innovative features of the Galaxy 
Classroom. We are interested in how helpful the following activities were in 
encouraging students to read and write. 



Student Enthusiasm About Reading and Writing 





Very 

Enthusiastic 


Somewhat 

Enthusiastic 


Not 

Enthusiastic 


Did Not Do 1 


N=99 

N=100 

N=101 

N=100 

N=100 

N=99 

N=103 


Student faxing within your 
school 


53 

(53.5%) 


16 

(16.2%) 


4 

(4.0%) 


26 

(26.3%) 


Student faxing to partner/other 
school 


74: 

(74.0%) 


17 

(17.0%) 


2 

(2.0%) 


7 

(7.0%) 


Student faxing to THE HOUSE 


82 

(81.2%) 


14 

(14.6%) 


0 

(0.0%) 


5 

(5.0%) 


Student faxing to THE MAX 


64 

(64.0%) 


26 

(26.0%) 


3 

(3.0%) 


7 

(7.0%) 


Receiving faxes from within 
your school 


46 

(46.0%) 


23 

(23.0%) 


4 

(4.0%) 


27 

(27.0%) 


Receiving faxes from 
partner/ other school 


69 

(69.1%) 


9 

(9.1%) 


3 

(3.0%) 


18 

(18.2%) 


Seeing faxes on the B-Show 


83 

(80.6%) 


14 

(13.6%) 


3 

(29.%) 


3 

(2.9%) 


Receiving THE MAX 


53 

(52.0%) 


35 

(34.3%) 


11 

(10.8%) 


3 

(2.9%) 


N=99 
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5. Please rate the usefulness of the following in helping you to implement Galaxy with 
your students: 



I. Teacher Resource Materials Very Useful 


Somewhat 

Useful 


Not Very 
Useful 


Did Not Use 


N=101 

N=102 

N=101 

N=99 

N=102 

N=101 

N=101 

N=97 

N=98 


A. Theme folders 


Show synopses 


84 

(SS.-Z^/o) 


15 

(14.9%) 


1 

(1.07o) 


1 

(1.07o) 


List of critical issues 


83 

(81.4%) 


18 

(17.6%) 


1 

(1.0%) 


1 

(1.07o) 


Literacy strategies and 
demonstration 


87 

(86.1%) 


14 

(13.9%) 


0 

(0.07o) 


0 

(0.07o) 


Literary references 


75 

(75.8%) 


21 

(21.2%) 


2 

(2.0%) 


1 

(1.0%) 


Lesson plans /activities 


96 

(94.1%) 


5 

(4.9%) 


1 

(1.0%) 


0 

(0.07o) 


Written copy of the raps 


65 

(64.4%) 


31 

(30.7%) 


3 

(3.0%) 


2 

(2.07o) 


Theme bibliographies 


76 

(75.2%) 


24 

(23.8%) 


1 

(1.0%) 


0 

(0.07o) 


B. Other Materials 


Teacher Resource Book 


84 

(86.6%) 


11 

(11.3%) 


1 

(1.0%) 


1 

(1.07o) 


Literacy Activities Handbook 


81 

(82.7%) 


16 

(16.3%) 


0 

(0.0%) 


1 

(1.07o) 


II. Technical Aids : ; 7i.- 


Very Useful 


Somewhat 

Useful 


Not Very 
. Useful 


Did Not Use 


N=98 

N=98 


1-800 phone number for technology 
problems 


65 

(66.3%) 


12 

(12.2%) 


4 

(4.1%) 


17 

(17.37o) 


Galaxy technical manual (fax, VCR, 
etc.) 


45 

(44.6%) 


36 

(35.6%) 


12 

(11.9%) 


8 

{7.9%) 







Did Not Use 


m* Very Useful 


Somewhat 

Useful 


Not Very 
Useful 


Los Angeles Institute 


80 

(84.2%) 


8 

(8.4%) 


2 

(2.1%) 


5 

(5.37o) 


N=95 

N=96 


Atlanta Institute 


78 

(81.3%) 


11 

(11.5%) 


0 

(0.07o) 


7 

{7.3%) 


Teacher newsletter 


41 

(46.6%) 


30 

(34.17o) 


8 

(9.17o) 


9 

(10.27o) 


N=96 

N=86 


Technical teleconference (April) 


25 

(29.1%) 


21 

(24.4%) 


7 

(8.17o) 


33 

(38.47o) 
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6. In your opinion, how well does Galaxy reach the following students; 



Very well 


Somewhat 

well 


Not very 
well 


Not 

applicable 


N=105 

N=104 

N=88 

N=80 

N=82 

N=99 

N=101 

N=97 

N=99 

N=88 

N=96 


Boys 


95 

(90.5%) 


10 

(9.5%) 


0 

(0.0%) 


0 


Girls 


98 

(94.2%) 


6 

(5.8%) 


0 

(0.0%) 


0 


Spanish LEP 


41 

(91.1%) 


4 

(8.9%) 


0 

(0.0%) 


43 


Other LEP 


19 

(23.8%) 


11 

(13.8%) 


50 

(62.5%) 


0 


New immigrants 


20 

(66.7%) 


9 

(30.0%) 


1 

(3.3%) 


52 


High-performing 


91 

(94.8%) 


5 

(5.2%) 


0 

(0.0%) 


3 


Low-performing 


86 

(85.1%) 


13 

(12.9%) 


2 

(2.0%) 


0 


Multicultural populations 


83 

(93.3%) 


6 

(6.7%) 


0 

(0.0%) 


8 


Low socio-economic background 


92 

(92.9%) 


6 

(6.1%) 


1 

(1.0%) 


0 


Hearing-impaired 


27 

(93.1%) 


2 

(6.9%) 


0 

(0.0%) 


59 


Learning-disabled 


52 

(81.3%) 


12 

(18.8%) 


0 

(0.0%) 


32 


Behavior-disordered 


55 

(77.8%) 


16 

(22.2%) 


1 

(1.4%) 


24 


N=96 


Other special education 


36 

(83.7%) 


7 

(16.3%) 


0 

(0.0%) 


38 


N=91 
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7. In your opinion, how well does Galaxy reach students in the following groups: 





Very well 


Somewhat 

well 


Not very 
well 


Not 

applicable 


White (non-Hispanic) 


76 

(91.6%) 


5 

(6.0%) 


2 

(2.4%) 


16 


Hispanic 


66 

(97.1%) 


2 

(2.9%) 


0 

(0.0%) 


29 


African American 


89 

(94.7%) 


5 

(5.3%) 


0 

(0.0%) 


9 


Asian or Pacific Islanders 


36 

(76.6%) 


11 

(23.4%) 


0 

(0.0%) 


46 


Native American 


29 

(90.6%) 


3 

(9.4%) 


0 

(0.0%) 


57 


Other groups 


1 

(100.0%) 


0 

(0.0%) 


0 

(0.0%) 


31 



N=99 



N=97 



N=103 



N=93 



N=89 



N=32 



8. How much has Galaxy helped you to do the following: 













Recognize individual strengths 


52 

(51.55%) 


41 

(40.6%) 


6 

(5.9%) 


2 

(2.0%) 


Enhance students’ self-esteem 


70 

(68.6%) 


28 

(27.5%) 


3 

(2.9%) 


1 

(1.0%) 


Turn students on to school 


67 

(66.3%) 


28 

(27.7%) 


4 

(4.0%) 


2 

(2.0%) 


Enable students to value cultural 
diversity 


57 

(55.9%) 


38 

(37.3%) 


6 

(5.9%) 


1 

(1.0%) 


Develop a more meaningful 
relationship with your students 


60 

(59.4%) 


29 

(28.7%) 


7 

(6.9%) 


5 

(5.0%) 



N=101 



N=102 



N=101 



N=102 



N=101 
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9. How well does Galaxy help students to: 








VeryWah 


Somewhat 

W«fl 


Not Very 
Well 


Not At All 


N=102 

N=102 

N=101 

N=99 

N=101 

N=101 

N=101 

N=101 

N=101 

N=101 

N=100 


Bra^'iStonn (cluster) to generate 
ideas for writing 


79 

(77.5%) 


23 

(22.5%) 


0 

(0.0%) 


0 

(0.0%) 


Write more frequently 


84 

(82.4%) 


15 

(14.7%) 


1 

(1.0%) 


2 

(2.0%) 


Revise writing for meaning 


38 

(37.6%) 


55 

(54.5%) 


6 

(5.9%) 


2 

(2.03 \ 


Reflect on and respond to what 
they read and write 


61 

(61.6%) 


36 

(36.4%) 


1 

(1.0%) 


1 

(1.0%) 


Write for "real" audiences 


77 

(76.2%) 


23 

(22.8%) 


1 

(1.0%) 


0 

(0.0%) 


Debate issues, using evidence 


58 

57.4% 


40 

(39.6%) 


3 

(3.0%) 


0 

(0.0%) 


Engage in critical thinking 


71 

(70.3%) 


30 

(29.7%) 


0 

(0.0%) 


0 

(0.0%) 


View themselves as confident, 
competent users of language 


54 

(53.5%) 


43 

(42.6%) 


4 

(4.0%) 


0 

(0.0%) 


See an issue from multiple 
perspectives 


57 

(66.3%) 


32 

(31.7%) 


2 

(2.0%) 


0 

(0.0%) 


Make meaning from text 
(predictions and context) 


65 

(64.4%) 


35 

(34.7%) 


1 

(1.0%) 


0 

(0.0%) 


Make mecining from other media 


67 

(67.0%) 


29 

(29.0%) 


4 

(4.0%) 


0 

(0.0%) 


Work productively in collaborative 
groups 


58 

(57.4%) 


42 

(41.6%) 


1 

(1.0%) 


0 

(0.0%) 


N=101 


Relate the Galaxy themes to then* 
own experience 


85 

(84.2%) 


15 

(14.9%) 


1 

(1.0%) 


0 

(0.0%) 


N=101 


Develop an in-depth understanding 
of the seven themes 


59 

(58.4%) 


39 

(38.6%) 


3 

(3.0%) 


0 

(0.0%) 


N=101 
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10. Since Galaxy began, my students: 



True for 
Most 


True For 
Some 


True for a 
Few 


Not True 
for Any 


'j^iRank of 
Importancerf 


Have a better attitude about 
reading and writing 


59 

(59.0%) 


32 

(32.0%) 


9 

(9.0%) 


0 

(0.0%) 


42 


Are more willing to undertake 
writing assignments 


57 

(54.8%) 


39 

(36.5%) 


6 

(5.8%) 


2 

(1.9%) 


40 


Do a better job of working with 
their peers to solve problems 


37 

(35.9%) 


49 

(47.6%) 


15 

(14.6%) 


2 

(1.9%) 


31 


Read more 


53 

(52.0%) 


31 

(29.8%) 


18 

(17.3%) 


2 

(1.9%) 


30 


Are more willing to write on 
their own 


53 

(51.0%) 


40 

(38.5%) 


8 

(7.7%) 


3 

(2.9%) 


29 


Speak up more in class 
discussion 


49 

(47.1%) 


46 

(44.2%) 


8 

(7.7%) 


1 

(1.0%) 


25 


Have a better attitude about 
school 


55 

(52.9%) 


33 

(31.7%) 


14 

(13.5%) 


2 

(1.9%) 


23 


Reason about issues with their 
peers 


40 

(38.8%) 


43 

(41.7%) 


19 

(18.4%) 


1 

(1.0%) 


18 


Misbehave less during Galaxy 
than at other times 


61 

(59.2%) 


30 

(29.1%) 


11 

(10.7%) 


1 

(1.0%) 


17 


Misbehave in class in general 
less than before Galaxy 


17 

(17.3%) 


50 

(51.07o) 


23 

(23.5%) 


8 

(8.2%) 


6 


Are absent less than they were 
before Galaxy 


24 

(24.7%) 


30 

(30.9%) 


20 

(20.6%) 


23 

(23.7%) 


4 


Complete Galaxy homework 
more than other subjects 


39 

(39.8%) 


34 

(34.7%) 


16 

(16.3%) 


9 

(9.2%) 


3 


Complete homework in general 
more often 


14 

(14.0%) 


48 

(48.0%) 


25 

(25.0%) 


13 

(13.0%) 


1 



N=100 



N=104 



N=103 



N=104 



N=104 



N=104 



N=104 



N=103 



N=103 



N=98 



N=97 



N=98 



N=100 



* Based on teacher selection of the three most important behaviors 
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